DataFusion¶

Driver Version v0.24.1 Tested With Apache DataFusion 53

This driver provides access to Apache DataFusion .

Note

This project is not part of the Apache Software Foundation.

Installation & Quickstart¶

The driver can be installed with dbc :

dbc install datafusion

Connecting¶

DataFusion is an in-process query engine and does not need a connection string or URI.

from adbc_driver_manager import dbapi

dbapi.connect(driver="datafusion")

You can provide the URI datafusion://, but no other URI is currently accepted:

from adbc_driver_manager import dbapi

dbapi.connect("datafusion://")

Note: The example above is for Python using the adbc-driver-manager package but the process will be similar for other driver managers. See adbc-quickstarts .

Feature & Type Support¶

Feature		Apache DataFusion
Bulk Ingestion	Create	✅
	Append	✅
	Create/Append	✅
	Replace	✅
	Temporary Table	❌
	Target Catalog	✅
	Target Schema	✅
	Non-nullable fields are marked NOT NULL	❌
Catalog (GetObjects)	depth=catalogs	✅
	depth=db_schemas	✅
	depth=tables	✅
	depth=columns (all)	✅
Get Parameter Schema		✅
Get Table Schema		✅
Prepared Statements		✅
Transactions		❌

Types¶

Database to Arrow¶

Database Type	Apache DataFusion
BIGINT	int64
BOOLEAN	bool
BYTEA	binary
DATE	date32[day]
DECIMAL	decimal128
DOUBLE	double
FLOAT	float
INT	int32
SMALLINT	int16
TIME	time64[ns]
TIMESTAMP	timestamp[ns] [1]
TIMESTAMP WITH TIME ZONE	timestamp[ns] [1] [2]
VARCHAR	string, string_view ⚠️ [3]

Arrow to Database¶

Arrow Type	Apache DataFusion Type
	Bind	Ingest
	binary	BYTEA
binary_view	❌	BYTEA
bool	BOOLEAN
date32[day]	DATE
decimal128	NUMERIC	DECIMAL
double	DOUBLE PRECISION	DOUBLE
fixed_size_binary	❌	BYTEA
float	REAL	FLOAT
int16	SMALLINT
int32	INT
int64	BIGINT
large_binary	❌	BYTEA
large_string	❌	VARCHAR
string	❌	VARCHAR
string_view	❌	VARCHAR
time32[ms]	(NA/not tested)	TIME
time32[s]	(NA/not tested)	TIME
time64[ns]	TIME
time64[us]	(NA/not tested)	TIME
timestamp[ms]	TIMESTAMP(3)	TIMESTAMP
timestamp[ms] (with time zone)	❌	TIMESTAMP WITH TIME ZONE
timestamp[ns]	TIMESTAMP(9)	TIMESTAMP
timestamp[ns] (with time zone)	❌	TIMESTAMP WITH TIME ZONE
timestamp[s]	TIMESTAMP(0)	TIMESTAMP
timestamp[s] (with time zone)	❌	TIMESTAMP WITH TIME ZONE
timestamp[us]	TIMESTAMP(6)	TIMESTAMP
timestamp[us] (with time zone)	❌	TIMESTAMP WITH TIME ZONE

Compatibility¶

This driver was tested on:

Apache DataFusion 53.1.0