Connect Google BigQuery
Datarelix connects to BigQuery via a read-only service using the Google BigQuery API and GoogleSQL (also called Standard SQL).
Prerequisites
- A Google Cloud project (the billing project that will pay for queries).
- Access to the dataset(s) you want to analyze. BigQuery is unique in that the data project can differ from the billing project — e.g. querying
bigquery-public-data.*datasets. - For service-account auth: a service account with
BigQuery Data ViewerandBigQuery Job Userroles (Data Viewer on the selected dataset, Job User on the billing project). - For OAuth: a Google account that can grant the BigQuery scope (
https://www.googleapis.com/auth/bigquery).
The two-project model
BigQuery jobs run in your billing project but read from the data project:
- Billing project — set this on the connection (
Project IDfield). Query jobs run and are billed here. - Data project — specify it as the first segment of Allowed dataset, e.g.
bigquery-public-data.usa_names. Reading from another project needs no extra setup beyond the permissions below.
Connection form
Project ID (billing): your-billing-projectLocation: US (or your preferred region)Allowed dataset: data-project.dataset_nameAuth mode: select an available modeLocation must match the dataset’s region. BigQuery runs the query job in the location you set, and rejects it if that doesn’t match where the dataset lives — e.g. a dataset in
asia-northeast1queried withLocation: USfails with “Not found: Dataset … was not found in location US.” Set Location to the dataset’s region. Check each public dataset’s location rather than assuming it isUS. See BigQuery locations.
Auth modes
Service account JSON
A Google Cloud service account authenticates using a JSON key file. The key is stored encrypted. Use an identity approved for this connection; dashboard refresh remains manual.
Setup
- In the Google Cloud Console, create a service account (IAM & Admin → Service Accounts → Create Service Account).
- Give it a descriptive name (e.g.
datarelix-bq-reader). - On the Grant this service account access to project step, assign these two roles:
BigQuery Data Viewer(grant at the selected dataset, separately from project IAM)BigQuery Job User(on the billing project)
- After creation, create a JSON key for the service account (Keys → Add Key → Create new key → JSON). A
.jsonfile downloads automatically. - Open the file and copy its entire contents.
Where to find your credentials
- The JSON key file you downloaded in step 4 above.
- If you already have a service account: IAM & Admin → Service Accounts → click the SA → Keys tab → add a new JSON key.
What to enter
Project ID (billing): your-billing-projectLocation: USAllowed dataset: your-project.your_datasetAuth mode: Service account JSONService account JSON: { "type": "service_account", "project_id": "...", ... }Paste the entire contents of the downloaded JSON file into the Service account JSON field.
OAuth
The user grants BigQuery access via Google’s consent screen. A refresh token is stored encrypted, scoped to this connection. Best for interactive analysts where you want per-user query attribution.
Authorizing a BigQuery connection is always a separate, explicit step. Signing in to Datarelix with Google grants identity only and never touches your data — the two are deliberately unconnected.
About the scope Google shows you. Google’s consent screen names the https://www.googleapis.com/auth/bigquery scope, whose description mentions managing your data. Datarelix requests this scope for BigQuery API calls; effective access still depends on the identity’s IAM permissions. Datarelix only ever issues SELECT: statements are validated before execution and non-read operations are rejected by the query service, not merely discouraged. If you want the restriction enforced by Google as well as by us, use service-account auth with BigQuery Data Viewer + BigQuery Job User and no write roles.
Setup
Confirm the required IAM grants before authorising — BigQuery access is controlled by the Google account’s IAM permissions. The account that authorizes the connection must have:
BigQuery Data Vieweron the selected dataset.BigQuery Job Useron the billing project.
Where to assign permissions
Google Cloud Console → IAM & Admin → IAM → find or add the user → assign the roles above.
What to enter
Project ID (billing): your-billing-projectLocation: USAllowed dataset: your-project.your_datasetAuth mode: OAuthAfter saving, Datarelix opens a Google consent screen popup. Sign in with the Google account that has the required IAM permissions. The connection is authorized once you approve the BigQuery scope.
Scope semantics
Select the approved dataset(s), using data-project.dataset_name identifiers. Queries are restricted to the selected datasets. Confirm that all queried datasets have a compatible location.
Discovery
Fully supported, including public datasets. Discovery lists the tables in each selected dataset and detects tables that require a partition filter, so generated queries include that filter automatically.
Limitations
- Cross-dataset queries within one connection: technically possible in BigQuery SQL via fully-qualified
project.dataset.tablereferences, but Datarelix restricts queries to the configured allowed datasets. - Partition-required tables: the partition filter is added automatically when table metadata is available. Re-run discovery if you add new partitioned tables.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
403 Access Denied | Missing BigQuery Data Viewer role | Grant on the data project in IAM & Admin → IAM. |
403 caller does not have permission (jobs) | Missing BigQuery Job User on billing project | Grant on the billing project. |
partitionFilter required | Querying a partition-required table without a filter | Re-run discovery so the partition column is picked up automatically. |
| OAuth popup closes without token | Browser blocked the popup | Allow popups for *.datarelix.ai. |
| Service account JSON rejected | Wrong format or truncated paste | Paste the entire contents of the .json file, including the outer { }. |
Per-query billing limit
The connector sets maximum_bytes_billed to 10 GiB (10 × 1024³ bytes) per query. This is not a monthly spending cap. Repeated questions and manual dashboard refreshes create more jobs. Under on-demand billing, a result LIMIT does not generally reduce scanned bytes; use relevant partition filters and selected columns. Capacity billing needs its own review. BigQuery cost controls
Service-account impersonation
The Service Account Impersonation auth mode takes a target service-account email; Datarelix’s own service identity then obtains short-lived credentials for that account, so no key is stored. It requires an agreed arrangement: contact Datarelix before selecting it. The target service account needs dataset read and billing-project job permissions, and Datarelix’s service identity needs permission to impersonate it.