Query Files & Object Storage
Datarelix can query data files — Delta Lake, Parquet, CSV, JSON, and Excel — directly from AWS S3, Azure Blob/ADLS, Google Cloud Storage, or a direct HTTPS URL, powered by an embedded, read-only DuckDB engine. No database server required.
Instead of schemas and tables, you define a list of datasets. Each dataset is a logical table
backed by a file, a folder of files, or a Delta table. Datarelix exposes each dataset as a
queryable table named however you choose — the AI plans SELECT … FROM your_table and never sees or
constructs the underlying file paths.
Prerequisites
- A storage location Datarelix can reach over the network.
- Credentials for that storage (or a public HTTPS URL).
- One or more files in a supported format: Parquet, CSV, JSON, Excel (.xlsx), or Delta Lake.
Connection form
| Field | Meaning |
|---|---|
| Storage provider | Selects the storage backend: AWS S3, Azure, GCS, or Direct HTTPS. |
| Datasets | One or more named tables. For each: a name, format, which files, and a path/URI. |
Datasets
For each dataset you provide:
- Table name — the identifier the AI queries (e.g.
sales). Letters, digits, and underscores. - Format —
parquet,csv,json,excel, ordelta. - Which files (non-Delta):
All files in directory(union),Latest file only, orA single file. - Path / URI — an exact file for
A single file; a directory or glob for the other modes (e.g.s3://bucket/sales/ors3://bucket/sales/*.parquet). - Version (Delta only): blank reads the latest version; set a number to pin a specific version.
- CSV options: header row toggle and delimiter. Excel: optional sheet name.
“Latest file” picks the newest file by name (descending sort), which matches date/sequence-stamped filenames such as
events-2026-07-01.parquet.
When All files or Latest file points at a bare directory (no * wildcard), a format-specific
pattern is applied automatically — s3://bucket/sales/ is read as s3://bucket/sales/*.parquet
for a Parquet dataset. Use an explicit glob when you need finer control.
Two combinations are not supported: Direct HTTPS URLs are always a single file (HTTP servers
can’t list directories), and Excel datasets read one file — use Latest file or A single file.
Auth modes
AWS S3
| Field | Notes |
|---|---|
| AWS Region | e.g. us-east-1. |
| Access Key ID / Secret | Optional — leave blank only for publicly readable objects. |
Use s3://bucket/path/... URIs.
Azure Blob / ADLS
| Field | Notes |
|---|---|
| Storage Account | Your Azure storage account name. |
| Account Key or SAS Token | One is required. |
Use az://container/path/... URIs.
Google Cloud Storage
| Field | Notes |
|---|---|
| HMAC Key ID / Secret | Create interoperability HMAC keys for your service account. |
Use gs://bucket/path/... URIs.
Direct HTTPS URL
| Field | Notes |
|---|---|
| Authorization Header | Optional — for token-protected URLs (e.g. Bearer abc123). Leave blank for public files. |
Use https://host/path/file.parquet URIs.
Query behavior
- Read-only. Only
SELECTqueries run. Mutations and IO/admin statements are rejected. - The AI references your dataset names only. File-reading functions (
read_parquet,delta_scan,glob, …) and arbitrary paths in generated SQL are rejected — paths come solely from your saved connection. - Results are capped at 5,000 rows per query.
Discovery
After connecting, discovery reads each dataset’s columns and types. Optional AI schema analysis adds table and column descriptions. Files carry no declared keys or foreign keys, so relationships are suggested from naming conventions.
Limitations & troubleshooting
- Delta version pinning — if the pinned version cannot be read, the dataset fails to load rather than falling back to another version. Clear Version to read the latest.
- “Latest file” is filename-based, not modification-time based.
- Access denied / cannot read file — check the credentials and that the path/glob matches actual objects. For S3, check that the access key you entered can read the bucket.
- Excel requires the DuckDB
excelextension; Delta requires thedeltaextension. Both are already installed — nothing to set up on your side.