BigQuery v1.0.0¶
This driver provides access to BigQuery , a data warehouse offered by Google Cloud.
Note
This project is not affiliated with Google.
Installation¶
The BigQuery driver can be installed with dbc :
dbc install bigquery
Pre-requisites¶
Using the BigQuery driver requires some setup before you can connect:
Create a Google Cloud account
Install the Google Cloud CLI (for managing credentials)
Authenticate with Google Cloud
Run
gcloud auth application-default login
Create, find, or reuse a project and dataset (record these for later)
Connecting¶
To connect, replace my-gcp-project and my-gcp-dataset below with the appropriate values for your situation and run the following:
from adbc_driver_manager import dbapi
conn = dbapi.connect(
driver="bigquery",
db_kwargs={
"adbc.bigquery.sql.project_id": "my-gcp-project",
"adbc.bigquery.sql.dataset_id": "my-gcp-datase"
}
)
Note: The example above is for Python using the adbc-driver-manager package but the process will be similar for other driver managers.
Feature & Type Support¶
| Feature | Support | |
|---|---|---|
| Bulk Ingestion | Create | ✅ |
| Append | ✅ | |
| Create/Append | ✅ | |
| Replace | ✅ | |
| Temporary Table | ❌ | |
| Specify target catalog | ❌ | |
| Specify target schema | ✅ | |
| Non-nullable fields are marked NOT NULL | ✅ | |
| Catalog (GetObjects) | depth=catalogs | ✅ |
| depth=db_schemas | ✅ | |
| depth=tables | ✅ | |
| depth=columns (all) | ✅ | |
| Get Parameter Schema | ❌ | |
| Get Table Schema | ✅ | |
| Prepared Statements | ✅ | |
| Transactions | ✅ | |
Types¶
BigQuery to Arrow¶
SQL Type |
Arrow Type |
|---|---|
ARRAY |
list [1] |
BOOL |
bool |
BOOLEAN |
bool |
BYTES |
binary |
DATE |
date32[day] |
DATETIME |
timestamp[us] [2] |
FLOAT64 |
double |
GEOGRAPHY |
extension<geoarrow.wkt> |
INT64 |
int64 |
INTERVAL |
month_day_nano_interval |
JSON |
extension<arrow.json> |
NUMERIC |
|
RANGE<DATE> |
struct<start: date32[day], end: date32[day]> |
RANGE<DATETIME> |
struct<start: timestamp[us], end: timestamp[us]> |
RANGE<TIMESTAMP> |
struct<start: timestamp[us, tz=UTC], end: timestamp[us, tz=UTC]> |
STRING |
string |
STRUCT |
struct |
TIME |
time64[us] |
TIMESTAMP |
timestamp[us, tz=UTC] [6] |
Arrow to BigQuery¶
| Arrow Type | BigQuery Type | |
|---|---|---|
| Bind | Ingest | |
| binary | BYTES | |
| bool | BOOLEAN | |
| date32[day] | DATE | |
| decimal128 | NUMERIC | |
| float | FLOAT64 | |
| double | FLOAT64 | |
| int16 | INT64 | |
| int32 | INT64 | |
| int64 | INT64 | |
| string | STRING | |
| time64[us] | TIME | |
| timestamp[us, tz=UTC] | TIMESTAMP | |
| timestamp[us] | DATETIME |
❌ [7] |