BigQuery Driver v1.10.0

Driver Version v1.10.0 Tested With BigQuery 1.72.0

This driver provides access to BigQuery , a data warehouse offered by Google Cloud.

Note

This project is not affiliated with Google.

Installation

The BigQuery driver can be installed with dbc :

dbc install bigquery

Pre-requisites

Using the BigQuery driver requires some setup before you can connect:

  1. Create a Google Cloud account

  2. Install the Google Cloud CLI (for managing credentials)

  3. Authenticate with Google Cloud

    • Run gcloud auth application-default login

  4. Create, find, or reuse a project and dataset (record these for later)

Connecting

To connect, replace my-gcp-project and my-gcp-dataset below with the appropriate values for your situation and run the following:

from adbc_driver_manager import dbapi

conn = dbapi.connect(
  driver="bigquery",
  db_kwargs={
      "adbc.bigquery.sql.project_id": "my-gcp-project",
      "adbc.bigquery.sql.dataset_id": "my-gcp-dataset"
  }
)

Note: The example above is for Python using the adbc-driver-manager package but the process will be similar for other driver managers.

The driver supports connecting with individual options or connection strings.

Connection String Format

BigQuery URI syntax:

bigquery://[Host]:[Port]/ProjectID?OAuthType=[AuthValue]&[Key]=[Value]&[Key]=[Value]...

The format follows a similar approach to the Simba BigQuery JDBC Connection String Format .

Components:

  • Scheme: bigquery:// (required)

  • Host: BigQuery API endpoint (optional, defaults to bigquery.googleapis.com)

  • Port: TCP port (optional, defaults to 443)

  • ProjectID: Google Cloud Project ID (required)

  • OAuthType: Authentication type number (optional, defaults to 0)

    • 0 - Application Default Credentials (default)

    • 1 - Service Account JSON File

    • 2 - Service Account JSON String

    • 3 - User Authentication (OAuth)

  • Additional Parameters: Connection configuration as key-value pairs. For a complete list of available parameters, see the Simba BigQuery JDBC Connector Configuration Options .

Note

Reserved characters in URI elements must be URI-encoded. For example, @ becomes %40.

Examples:

  • bigquery:///my-project-123 (uses Application Default Credentials)

  • bigquery://bigquery.googleapis.com/my-project-123?OAuthType=1&AuthCredentials=/path/to/key.json

  • bigquery:///my-project-123?OAuthType=3&AuthClientId=123.apps.googleusercontent.com&AuthClientSecret=secret&AuthRefreshToken=token

  • bigquery://bigquery.googleapis.com/my-project-123?OAuthType=0&DatasetId=analytics&Location=US

  • bigquery:///my-project-123?OAuthType=2&AuthCredentials=%7B%22type%22%3A%22service_account%22...%7D&DatasetId=data_warehouse

Feature & Type Support

Feature Support
Bulk Ingestion Create
Append
Create/Append
Replace
Temporary Table
Specify target catalog
Specify target schema
Non-nullable fields are marked NOT NULL
Catalog (GetObjects) depth=catalogs
depth=db_schemas
depth=tables
depth=columns (all)
Get Parameter Schema
Get Table Schema
Prepared Statements
Transactions

Types

BigQuery to Arrow

BigQuery Type

Arrow Type

ARRAY

list [1]

BIGNUMERIC

decimal256(76, 38)

BOOL

bool

BOOLEAN

bool

BYTES

binary

DATE

date32[day]

DATETIME

timestamp[us] [2]

FLOAT64

double

GEOGRAPHY

extension<geoarrow.wkt>

INT64

int64

INTERVAL

month_day_nano_interval

JSON

extension<arrow.json>

NUMERIC

decimal128(38, 9) [3] [4] [5]

RANGE<DATE>

struct<start: date32[day], end: date32[day]>

RANGE<DATETIME>

struct<start: timestamp[us], end: timestamp[us]>

RANGE<TIMESTAMP>

struct<start: timestamp[us, tz=UTC], end: timestamp[us, tz=UTC]>

STRING

string

STRUCT

struct

TIME

time64[us]

TIMESTAMP

timestamp[us] (with time zone) [6]

Arrow to BigQuery

Arrrow Type BigQuery Type
Bind Ingest
binary

BYTES

binary_view

BYTES

[9]

bool

BOOLEAN

date32[day]

DATE

decimal128

NUMERIC

double

FLOAT64

fixed_size_binary

BYTES

float

(not tested)

FLOAT64

int16

(not tested)

INT64

int32

(not tested)

INT64

int64

INT64

large_binary

BYTES

large_string

STRING

list

(not tested)

ARRAY [10] [11] [12]

string

STRING

string_view

STRING

[13]

struct

(not tested)

STRUCT

time32[ms]

TIME

time32[s]

TIME

time64[ns]

[7]

[14]

time64[us]

TIME

timestamp[ms]

DATETIME

[15]

timestamp[ms] (with time zone)

TIMESTAMP

[15]

timestamp[ns]

DATETIME [8]

[15]

timestamp[ns] (with time zone)

TIMESTAMP

[15]

timestamp[s]

DATETIME

[15]

timestamp[s] (with time zone)

TIMESTAMP

[15]

timestamp[us]

DATETIME

[15]

timestamp[us] (with time zone)

TIMESTAMP

Previous Versions

To see documentation for previous versions of this driver, see the following: