---
title: "Query Files & Object Storage"
description: "Query Delta, Parquet, CSV, JSON, and Excel files directly from S3, Azure, GCS, or a URL."
canonical: https://docs.datarelix.ai/guides/connections/duckdb-files/
---

# Query Files & Object Storage

Datarelix can query **data files** — Delta Lake, Parquet, CSV, JSON, and Excel — directly from
**AWS S3, Azure Blob/ADLS, Google Cloud Storage, or a direct HTTPS URL**, powered by an embedded,
read-only [DuckDB](https://duckdb.org/) engine. No database server required.

Instead of schemas and tables, you define a list of **datasets**. Each dataset is a logical table
backed by a file, a folder of files, or a Delta table. Datarelix exposes each dataset as a
queryable table named however you choose — the AI plans `SELECT … FROM your_table` and never sees or
constructs the underlying file paths.

## Prerequisites

- A storage location your Datarelix deployment can reach over the network.
- Credentials for that storage (or a public HTTPS URL).
- One or more files in a supported format: **Parquet, CSV, JSON, Excel (.xlsx), or Delta Lake**.

## Connection form

| Field | Meaning |
|-------|---------|
| **Storage provider** | Selects the storage backend: AWS S3, Azure, GCS, or Direct HTTPS. |
| **Datasets** | One or more named tables. For each: a **name**, **format**, **which files**, and a **path/URI**. |

### Datasets

For each dataset you provide:

- **Table name** — the identifier the AI queries (e.g. `sales`). Letters, digits, and underscores.
- **Format** — `parquet`, `csv`, `json`, `excel`, or `delta`.
- **Which files** (non-Delta): `All files in directory` (union), `Latest file only`, or `A single file`.
- **Path / URI** — an exact file for `A single file`; a directory or glob for the other modes
  (e.g. `s3://bucket/sales/` or `s3://bucket/sales/*.parquet`).
- **Version** (Delta only): blank reads the latest version; set a number to pin a specific version.
- **CSV options**: header row toggle and delimiter. **Excel**: optional sheet name.

> "Latest file" picks the newest file by name (descending sort), which matches date/sequence-stamped
> filenames such as `events-2026-07-01.parquet`.

When `All files` or `Latest file` points at a bare directory (no `*` wildcard), a format-specific
pattern is applied automatically — `s3://bucket/sales/` is read as `s3://bucket/sales/*.parquet`
for a Parquet dataset. Use an explicit glob when you need finer control.

Two combinations are not supported: **Direct HTTPS URLs** are always a single file (HTTP servers
can't list directories), and **Excel** datasets read one file — use `Latest file` or `A single file`.

## Auth modes

### AWS S3

| Field | Notes |
|-------|-------|
| AWS Region | e.g. `us-east-1`. |
| Access Key ID / Secret | Optional — leave blank only for publicly readable objects. |

Use `s3://bucket/path/...` URIs.

### Azure Blob / ADLS

| Field | Notes |
|-------|-------|
| Storage Account | Your Azure storage account name. |
| Account Key **or** SAS Token | One is required. |

Use `az://container/path/...` URIs.

### Google Cloud Storage

| Field | Notes |
|-------|-------|
| HMAC Key ID / Secret | Create [interoperability HMAC keys](https://cloud.google.com/storage/docs/authentication/hmackeys) for your service account. |

Use `gs://bucket/path/...` URIs.

### Direct HTTPS URL

| Field | Notes |
|-------|-------|
| Authorization Header | Optional — for token-protected URLs (e.g. `Bearer abc123`). Leave blank for public files. |

Use `https://host/path/file.parquet` URIs.

## Query behavior

- **Read-only.** Only `SELECT` queries run. Mutations and IO/admin statements are rejected.
- The AI references your **dataset names** only. File-reading functions (`read_parquet`, `delta_scan`,
  `glob`, …) and arbitrary paths in generated SQL are rejected — paths come solely from your saved
  connection.
- A per-query **row cap** (default 5000) is enforced.

## Discovery

After connecting, Datarelix introspects each dataset's columns (via `DESCRIBE`) and can optionally
use an LLM to add table/column descriptions. Files carry no declared keys or foreign keys, so
relationships are inferred by the LLM from naming conventions.

## Limitations & troubleshooting

- **Delta version pinning** requires a DuckDB build whose Delta extension supports version selection;
  otherwise the latest version is read.
- **"Latest file"** is filename-based, not modification-time based.
- **Access denied / cannot read file** — check the credentials and that the path/glob matches actual
  objects. For S3, check that the access key you entered can read the bucket.
- **Excel** requires the DuckDB `excel` extension; **Delta** requires the `delta` extension. Both are
  already installed — nothing to set up on your side.