Databricks

datarelix.ai connects to Databricks through a read-only query service. Queries run against a SQL Warehouse — not a general-purpose compute cluster.

Prerequisites

A Databricks workspace with Unity Catalog enabled.
A SQL Warehouse (Serverless or Pro). Classic warehouses also work but are slower to start.
A service principal or user with USE CATALOG, USE SCHEMA, and SELECT grants on the target catalog and schema.

Finding your connection details

See Get connection details for a Databricks compute resource for the official walkthrough. In the Databricks workspace UI:

Go to SQL → SQL Warehouses in the left sidebar.
Click your warehouse.
Open the Connection details tab.
Copy:
- Server hostname — three cloud-specific shapes:
  - AWS: dbc-12345678-abcd.cloud.databricks.com
  - Azure: adb-1234567890123456.7.azuredatabricks.net
  - GCP: <workspace-id>.<region>.gcp.databricks.com
- HTTP path — looks like /sql/1.0/warehouses/abc123def456.

The hostname maps to Workspace host and the path maps to HTTP path on the connection form.

Connection form

Workspace host:   dbc-12345678-abcd.cloud.databricks.com
HTTP path:        /sql/1.0/warehouses/abc123def456
Catalog:          main
Allowed schema:   default
Auth mode:        one of the three below

Auth modes

Personal Access Token (PAT)

Simplest path. Best for unattended/service use. A PAT is a long-lived token tied to a user or service principal; it’s stored encrypted on the connection.

Setup — create a PAT scoped to a service principal

In Databricks, go to Settings → Identity and access → Service principals.
Create or pick a service principal.

Grant it the required Unity Catalog privileges via a SQL Warehouse query:

GRANT USE CATALOG ON CATALOG main TO `your-sp-app-id`;
GRANT USE SCHEMA ON SCHEMA main.default TO `your-sp-app-id`;
GRANT SELECT ON SCHEMA main.default TO `your-sp-app-id`;

Generate a personal access token for that service principal: Settings → Identity and access → Service principals → [your SP] → Generate token.

Don’t use your personal PAT for shared service connections — when you leave, the connection breaks. Always bind to a service principal.

Where to find your credentials

PAT: Settings → Identity and access → Service principals → [your SP] → Tokens (or User settings → Developer → Access tokens for personal tokens)

What to enter

Workspace host:  dbc-12345678-abcd.cloud.databricks.com
HTTP path:       /sql/1.0/warehouses/abc123def456
Catalog:         main
Allowed schema:  default
Auth mode:       Personal Access Token
Token:           dapi••••••••••••••••

OAuth M2M (service principal)

OAuth machine-to-machine (M2M) client credentials grant. Better for production — tokens are short-lived and rotated automatically. Requires a client ID and client secret stored encrypted on the connection.

Setup

In Databricks, go to Settings → Identity and access → Service principals.
Create a service principal if you don’t have one.
Grant it Unity Catalog privileges (same GRANT SQL as above).
Go to [your SP] → Secrets → Generate secret. Copy the Client ID (the SP’s application ID) and the Secret value immediately — it’s only shown once.

Where to find your credentials

Field	Where to get it
Client ID	Settings → Identity and access → Service principals → [your SP] → Application ID
Client secret	[your SP] → Secrets → the value shown when you generated the secret

What to enter

Workspace host:  dbc-12345678-abcd.cloud.databricks.com
HTTP path:       /sql/1.0/warehouses/abc123def456
Catalog:         main
Allowed schema:  default
Auth mode:       OAuth M2M
Client ID:       your-sp-application-id
Client secret:   ••••••••

Federated JWT (SSO)

Binds the connection to the currently signed-in user. The orchestrator mints a workspace-scoped JWT from the user’s session and exchanges it for a Databricks token via the workspace’s OAuth token federation. No long-lived secret stored.

Requires:

A Databricks workspace with OIDC federation configured against your identity provider.
The signed-in user has CAN USE on the SQL Warehouse.

This is the default auth mode for SSO tenants where the workspace is already federated.

What to enter

Workspace host:  dbc-12345678-abcd.cloud.databricks.com
HTTP path:       /sql/1.0/warehouses/abc123def456
Catalog:         main
Allowed schema:  default
Auth mode:       Federated JWT

No credentials to enter — the connection uses the signed-in user’s identity automatically.

Unity Catalog namespace

Databricks uses Unity Catalog’s three-level namespace: catalog.schema.table.

Catalog — top-level namespace. Most workspaces use main; larger orgs split into dev, staging, prod, or by domain.
Schema — second level, often called “database” in the Spark UI for historical reasons. Maps to the connection’s Allowed schema field.
Table — the actual table or view.

To analyze multiple schemas, create one connection per schema.

Scope semantics

Catalog (default main) — top-level namespace.
Allowed schema (default default) — the schema the connection is restricted to. The planner blocks any query that references a schema outside the allowlist.

Discovery

Fully supported on Unity Catalog. The introspector reads from <catalog>.information_schema — tables, columns, table constraints, and key column usage. PK/FK constraints are detected when declared; many Unity Catalog schemas don’t declare them, in which case the LLM enrichment pass infers relationships from column-name conventions.

Unity Catalog is required. The legacy hive_metastore catalog has no information_schema, so discovery against it returns nothing. Point the connection at a Unity Catalog catalog (e.g. main), not hive_metastore. Migrate legacy tables to Unity Catalog if you need to analyze them.

Limitations

Unity Catalog only — the legacy hive_metastore catalog is not supported for discovery (no information_schema).
SQL Warehouse only — general-purpose compute clusters (notebooks, jobs) are not supported.
Read-only — the validator rejects write and DDL statements.
Cold starts — Serverless warehouses resume in ~5 s; Pro in ~30 s; Classic in 2–5 min. The first query after idle pays this cost.
Row limit — server-side cap of 5 000 rows by default.

Troubleshooting

Symptom	Likely cause	Fix
`Warehouse is not running`	Warehouse stopped and auto-resume off	Enable auto-resume on the warehouse, or warm it manually.
`403 PERMISSION_DENIED` on `USE CATALOG`	Service principal lacks Unity Catalog grants	Re-run the `GRANT USE CATALOG / SCHEMA / SELECT` SQL above.
`403 INVALID_HTTP_PATH`	Wrong HTTP path	Verify in SQL Warehouses → Connection details; path begins with `/sql/1.0/warehouses/`.
`OAuth client not authorized for scope sql`	M2M SP missing the `sql` scope	Re-issue the secret with `Scopes: all-apis` (or explicit `sql`) on creation.
`Token expired` (PAT)	PAT lifetime ended	PATs are time-limited; rotate via the Databricks admin API or SP token settings.
Discovery returns 0 tables	Catalog has no tables in the allowed schema	Check `SHOW TABLES IN <catalog>.<schema>` from a Databricks SQL editor.