adbc-driver-quack

An Apache Arrow ADBC driver for DuckDB's Quack remote protocol.

Returns Apache Arrow RecordBatches directly from a remote DuckDB server speaking Quack. Supports the standard ADBC bulk-ingest path (Statement.BindStream → APPEND_REQUEST) for fast column-oriented loads.

Distributed as:

• a Go module — github.com/gizmodata/adbc-driver-quack
• a pip install adbc-driver-quack wheel for Python (macOS / Linux / Windows × x64 / arm64)

Status: Alpha — v0.1.0-alpha.1 is the first release. The companion gizmodata/quack-jdbc JDBC driver is the same protocol from the JVM and is at v0.1.0-alpha.1 on Maven Central.

Quickstart

1. Start a Quack server (any DuckDB v1.5.3+)

-- in any DuckDB session — Quack is a core extension as of v1.5.3,
-- so no `core_nightly` repository or `-unsigned` flag is needed.
INSTALL quack;
LOAD quack;
CALL quack_serve('quack:localhost:9494', token=>'my-secret-token');

The server stays running until the DuckDB session exits. Press Ctrl-C in the DuckDB REPL to stop it.

Note: quack_serve accepts shorter forms — 'quack:localhost' uses the default port, and a bare quack_serve() with no first arg uses localhost as the host. We keep the explicit localhost:9494 form throughout this README so the client-side URI maps obviously to what the server is bound to.

If localhost ever gives you a connection refused (rare, but it can happen on a system whose /etc/hosts is set up such that the server binds one address family and the client dials the other), use 127.0.0.1 on both sides.

2. Install the driver

Python:

pip install adbc-driver-quack

Go:

go get github.com/gizmodata/adbc-driver-quack@latest

3. Connect and query

import adbc_driver_quack.dbapi as quack
import pyarrow

with quack.connect(
    uri="quack://localhost:9494",
    db_kwargs={"adbc.quack.token": "my-secret-token"},
) as conn, conn.cursor() as cur:
    cur.execute("SELECT 42 AS answer, 'hello duckdb' AS greeting")
    table: pyarrow.Table = cur.fetch_arrow_table()
    print(table)

The result is a real pyarrow.Table — pass it straight to Polars, Pandas, DuckDB-in-process, ibis, or anything else that consumes Arrow:

import polars as pl
df = pl.from_arrow(table)

Alternative: drive adbc_driver_manager directly

If you prefer the adbc-quickstarts idiom — passing the driver to adbc_driver_manager.dbapi.connect rather than going through our wrapper — point at the bundled shared library via _driver_path():

from adbc_driver_manager import dbapi
import adbc_driver_quack

with dbapi.connect(
    driver=adbc_driver_quack._driver_path(),
    entrypoint="QuackDriverInit",
    db_kwargs={
        "uri": "quack://localhost:9494",
        "adbc.quack.token": "my-secret-token",
    },
) as conn, conn.cursor() as cur:
    cur.execute("SELECT 42 AS answer")
    table = cur.fetch_arrow_table()

Both styles work the same on the wire — pick whichever reads better for your codebase.

Streaming large result sets

Cursor.fetch_record_batch() returns a pyarrow.RecordBatchReader that pulls one server-side DataChunk per read_next_batch() call. Memory stays bounded by the server's chunk size (~2k rows) even when the result is millions of rows:

with conn.cursor() as cur:
    cur.execute("SELECT * FROM lineitem")  # arbitrary size
    reader = cur.fetch_record_batch()
    for batch in reader:
        process(batch)  # one ~2k-row Arrow batch at a time

Bulk ingest (Arrow → DuckDB)

import pyarrow as pa
import adbc_driver_quack.dbapi as quack

table = pa.table({"id": [1, 2, 3], "name": ["alice", "bob", "carol"]})
with quack.connect(
    uri="quack://localhost:9494",
    db_kwargs={"adbc.quack.token": "my-secret-token"},
    autocommit=True,  # ADBC connections are autocommit-OFF by default;
                      # opt in here so the ingest persists on close
) as conn, conn.cursor() as cur:
    # create_append: create "customers" from the Arrow schema if it
    # doesn't exist, then append. One APPEND_REQUEST per RecordBatch.
    cur.adbc_ingest(table_name="customers", data=table, mode="create_append")

Heads-up — autocommit is off by default. Per the Python DB-API, quack.connect() opens connections inside a transaction. Without the autocommit=True above (or an explicit conn.commit()), the CREATE + append run in a transaction that is rolled back when the connection closes — adbc_ingest still returns the row count it sent, but nothing persists. Prefer explicit transactions? Drop autocommit=True and call conn.commit() after adbc_ingest():

with quack.connect(uri="quack://localhost:9494",
                    db_kwargs={"adbc.quack.token": "my-secret-token"}) as conn, conn.cursor() as cur:
    cur.adbc_ingest(table_name="customers", data=table, mode="create_append")
    conn.commit()  # without this, the ingest is rolled back on close

mode accepts the four standard ADBC ingest modes:

Table DDL for the create-family modes is generated from the Arrow schema. Pass db_schema_name=... to target a non-default schema.

Transactions (autocommit off)

import adbc_driver_quack.dbapi as quack

with quack.connect(
    uri="quack://localhost:9494",
    db_kwargs={"adbc.quack.token": "..."},
    autocommit=False,
) as conn, conn.cursor() as cur:
    cur.execute("INSERT INTO orders VALUES (1, 'pending')")
    cur.execute("INSERT INTO order_items VALUES (1, 'widget', 2)")
    conn.commit()  # both inserts persist atomically

Connection URL

quack://host[:port]

Token precedence matches quack-jdbc: an explicit adbc.quack.token (or password) wins, then adbc.quack.token_env, then adbc.quack.token_file. The env/file indirections are accepted only as ADBC options, never as quack://...?tokenEnv=... URL query parameters — a pasted URL must not be able to read a local secret and send it to whatever host the URL names.

The URI is its own kwarg; everything else goes through db_kwargs:

import adbc_driver_quack.dbapi as quack

quack.connect(
    uri="quack://localhost:9494",
    db_kwargs={
        "adbc.quack.token": "my-secret-token",
        "adbc.quack.tls": "false",
    },
)

Extra HTTP headers

Gateways and load balancers in front of a Quack server often need their own auth. adbc.quack.http.header.<Name> options add static headers to every request the driver makes (mirroring the EXTRA_HTTP_HEADERS parameter of DuckDB's own quack secret):

quack.connect(
    uri="quack://gateway.example.com:443",
    db_kwargs={
        "adbc.quack.tls": "true",
        "adbc.quack.token": "my-secret-token",
        "adbc.quack.http.header.X-Proxy-Authorization": "Bearer abc123",
    },
)

Like the token indirections, header options are rejected as URL query parameters — a pasted URL cannot inject headers into your requests. The protocol-owned headers (Content-Type, Accept, Host, Content-Length) are reserved and cannot be overridden.

Connection profiles & driver manifests

ADBC connection profiles (adbc-driver-manager ≥ 1.11) let you keep a connection's driver + options in a reusable TOML file instead of code. Profiles resolve the driver by name, which requires a driver manifest on the search path. Install ours once per environment:

$ python -m adbc_driver_quack install-manifest
Wrote ADBC driver manifest: .../etc/adbc/drivers/quack.toml

(Inside a virtualenv/conda env this targets the environment's auto-searched etc/adbc/drivers/; otherwise the per-user ADBC config directory. --user, --venv, and --dir PATH override; the same is available programmatically as adbc_driver_quack.install_manifest().)

With the manifest in place, the driver manager finds the driver by name — no import of adbc_driver_quack needed:

from adbc_driver_manager import dbapi

# Resolve by URI scheme alone:
conn = dbapi.connect(uri="quack://localhost:9494")

And a profile bundles the whole connection. Drop this in ~/.config/adbc/profiles/quack_prod.toml (Linux; ~/Library/Application Support/ADBC/Profiles/ on macOS, or any directory named in ADBC_PROFILE_PATH):

profile_version = 1
driver = "quack"

[Options]
uri = "quack://prod.example.com:9494"
"adbc.quack.tls" = true
"adbc.quack.token" = "{{ env_var(QUACK_TOKEN) }}"

then connect from any ADBC driver-manager binding:

conn = dbapi.connect(profile="quack_prod")

The {{ env_var(...) }} substitution keeps secrets out of the file; options set explicitly in code still override profile values.

Why ADBC and not JDBC?

Both drivers speak the same protocol to the same kind of server. Pick the one that fits your runtime:

Repo layout

adbc-driver-quack/
├── go.mod, go.sum
├── internal/
│   ├── codec/       — BinaryReader/Writer for DuckDB BinarySerializer
│   ├── quacktype/   — Logical / physical / extra type system + codec
│   ├── message/     — DataChunk, DecodedVector, MessageCodec, VectorCodec
│   └── transport/   — QuackURI parser + net/http transport (IPv4/IPv6 fallback)
├── driver/quack/    — pure-Go ADBC Driver/Database/Connection/Statement impl
├── pkg/quack/       — cgo c-shared wrapper (produces libadbc_driver_quack.{so,dylib,dll})
├── python/          — Python wheel sources (adbc_driver_quack)
└── .github/         — CI: go test, python tests, cibuildwheel matrix, PyPI publish

The internal/ layer is a clean-room Go port of the matching Java packages in quack-jdbc.

Credits

• Wire format: DuckDB Quack protocol
• Codec design: ported from gizmodata/quack-jdbc (MIT), which itself was clean-room ported from tobilg/quack-protocol (MIT)
• ADBC framework: Apache Arrow ADBC (Apache-2.0)

License

MIT — see LICENSE for full attribution.