spanner-adbc

A Python ADBC driver for Google Cloud Spanner.

Query Spanner through DBAPI 2.0 and fetch Apache Arrow results for pandas, Polars, DuckDB, or PyArrow.

Install

Requires Python 3.11 or later. Wheels bundle the native driver; see Supported platforms for OS requirements.

pip install "spanner-adbc[dbapi]"  # includes PyArrow and pandas

For the low-level ADBC API without DataFrame dependencies, install spanner-adbc. The Python import name is spanner_adbc.

Quickstart

This example assumes an existing Singers table with SingerId and FirstName columns.

import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    with conn.cursor() as cur:
        cur.execute("SELECT SingerId, FirstName FROM Singers")
        df = cur.fetch_df()          # -> pandas.DataFrame

Use GoogleSQL @name parameters with cur.execute(...); ? placeholders are unsupported. fetchone() and fetchall() return Python rows; Arrow and DataFrame helpers are shown below.

Driver manifest (driver="spanner")

The package's spanner.connect() loads its bundled library directly. To use the generic adbc_driver_manager by driver name, install an ADBC driver manifest. Use a current driver manager; Python manifest support requires version 1.8.0 or later.

pip install --upgrade adbc-driver-manager
python -m spanner_adbc.manifest install

The equivalent console command is spanner-adbc-install-manifest.

import adbc_driver_manager.dbapi

with adbc_driver_manager.dbapi.connect(
    driver="spanner",
    uri="spanner:///projects/my-project/instances/my-instance/databases/my-db",
) as conn:
    ...

Current driver managers can also infer spanner from the URI scheme when driver is omitted.

• Re-run the installer after upgrading, reinstalling, or moving the environment: the manifest contains the library's absolute path.
• The default directory is <sys.prefix>/etc/adbc/drivers inside a virtual environment, or the platform's user configuration directory otherwise. python -m spanner_adbc.manifest path prints the target path.
• Use install --dir /path/to/drivers for a custom directory on ADBC_DRIVER_PATH.
• For a standalone shared library, edit the Driver.shared paths in the repository's spanner.toml.

Authentication

With no credential options, the driver uses Application Default Credentials: run gcloud auth application-default login locally, set GOOGLE_APPLICATION_CREDENTIALS, or use an attached service account on Google Cloud.

Pass explicit credentials in db_kwargs alongside DatabaseOptions.URI.value:

Use each enum member's .value as the key. ACCESS_TOKEN cannot be combined with key-file credentials or impersonation. See the option reference for scopes and other auth settings.

For the Spanner emulator, use anonymous mode:

import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

with spanner.connect(db_kwargs={
    DatabaseOptions.URI.value: "spanner:///projects/p/instances/i/databases/d",
    DatabaseOptions.ENDPOINT.value: "localhost:9010",
    DatabaseOptions.EMULATOR.value: "true",
}) as conn:
    ...

Emulator mode rejects explicit credential options; ambient ADC is ignored.

Options

Use DatabaseOptions, ConnectionOptions, and StatementOptions enums, or raw option keys. The option reference lists types, defaults, and accepted values. Set cursor options with conn.cursor(adbc_stmt_kwargs={...}) or cur.adbc_statement.set_options(**{...}).

import spanner_adbc.dbapi as spanner
from spanner_adbc import ConnectionOptions, DatabaseOptions, StatementOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/p/instances/i/databases/d"},
    conn_kwargs={ConnectionOptions.READ_STALENESS.value: "max:10s"},
    autocommit=True,  # bounded staleness on a single-use read
) as conn:
    with conn.cursor(
        adbc_stmt_kwargs={StatementOptions.ROWS_PER_BATCH.value: "1024"}
    ) as cur:
        cur.execute("SELECT * FROM Singers")

In manual query transactions, max:<duration> and min:<timestamp> bounds become exact staleness and a fixed read timestamp respectively. Set ConnectionOptions.READONLY.value to "true" to reject DML, DDL, and ingest; rollback remains available.

Transactions

DBAPI defaults to autocommit=False. A manual transaction accepts either queries or writes, chosen by its first query or write; mixing them raises adbc_driver_manager.ProgrammingError until commit or rollback. Queries share a snapshot. DML is buffered until conn.commit(), so queries cannot read buffered writes.

DDL always executes immediately: rollback cannot undo it, and it runs before buffered DML. Use autocommit=True for immediately committed DML, including THEN RETURN statements. See transactions for bulk-ingest and partitioned-DML behavior.

import spanner_adbc.dbapi as spanner
from adbc_driver_manager import ProgrammingError
from spanner_adbc import DatabaseOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    with conn.cursor() as cur:
        # DDL applies immediately.
        cur.execute("DROP TABLE IF EXISTS Albums")
        cur.execute("CREATE TABLE Albums (Id INT64 NOT NULL) PRIMARY KEY (Id)")

        cur.execute("INSERT INTO Albums (Id) VALUES (1)")  # a DML transaction: buffered
        # Commit before querying the inserted row.
        try:
            cur.execute("SELECT COUNT(*) FROM Albums")
            raise AssertionError("expected the guarded query to raise")
        except ProgrammingError:
            pass
    conn.commit()

    with conn.cursor() as cur:
        cur.execute("SELECT COUNT(*) FROM Albums")
        assert cur.fetchone()[0] == 1
    conn.rollback()  # end the query snapshot

Working with DataFrames

The quickstart uses pandas. The examples below use the same Singers table with SingerId and FirstName columns. Install polars or duckdb separately for those examples.

pyarrow — results as a native Arrow table:

import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    with conn.cursor() as cur:
        cur.execute("SELECT SingerId, FirstName FROM Singers ORDER BY SingerId")
        table = cur.fetch_arrow_table()      # -> pyarrow.Table

polars — read straight from the connection:

import polars as pl
import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    df = pl.read_database(
        "SELECT SingerId, FirstName FROM Singers ORDER BY SingerId",
        connection=conn,                     # an ADBC connection, not a URI
    )

DuckDB — query the fetched Arrow table in-process:

import duckdb
import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    with conn.cursor() as cur:
        cur.execute("SELECT SingerId, FirstName FROM Singers")
        singers = cur.fetch_arrow_table()

# DuckDB can query the Arrow table by variable name.
top = duckdb.sql("SELECT COUNT(*) AS n, MIN(FirstName) AS first FROM singers").fetchone()

Bulk insert a DataFrame

cur.adbc_ingest(table, data, mode=...) inserts Arrow-compatible data in bulk. Convert pandas DataFrames to Arrow as shown here:

import pandas as pd
import pyarrow as pa
import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions

frame = pd.DataFrame({"SingerId": [10, 11], "FirstName": ["Carol", "Dave"]})

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
    autocommit=True,                         # apply immediately; returns the row count
) as conn:
    with conn.cursor() as cur:
        # `append` inserts into an existing table (the default mode is `create`).
        rows = cur.adbc_ingest("Singers", pa.Table.from_pandas(frame), mode="append")

The mode selects how the target table is handled:

• create — create the table from the data's Arrow schema first, erroring if it already exists (the default).
• append — insert into an existing table.
• create_append — create the table only if it is absent, then insert.
• replace — drop any existing table, recreate it from the schema, then insert.

Create modes omit a primary key, so Spanner supplies a hidden rowid column. For an explicit primary key, create the table with SQL first and use mode="append". In manual mode, inserts need conn.commit(); table creation or replacement still happens immediately. Autocommit loads may commit in multiple chunks, so a failed load can leave rows written.

Partitioned reads and Data Boost

A large scan can be split into independent partitions and read in parallel — optionally on Spanner's serverless Data Boost compute, so the work is isolated from your provisioned instance. This uses the ADBC partitioned-execution extension (adbc_execute_partitions / adbc_read_partition):

import spanner_adbc.dbapi as spanner
from spanner_adbc import DatabaseOptions, StatementOptions

with spanner.connect(
    db_kwargs={DatabaseOptions.URI.value: "spanner:///projects/my-project/instances/my-instance/databases/my-db"},
) as conn:
    with conn.cursor() as cur:
        # Optional statement options, set on the underlying ADBC statement:
        cur.adbc_statement.set_options(**{
            StatementOptions.DATA_BOOST.value: "true",  # run on Data Boost
        })
        partitions, schema = cur.adbc_execute_partitions("SELECT SingerId FROM Singers")

    # Each descriptor is opaque bytes; it can be shipped to another worker,
    # process, or connection and read independently.
    for token in partitions:
        with conn.cursor() as cur:
            cur.adbc_read_partition(token)
            table = cur.fetch_arrow_table()
            ...

Spanner decides partitionability from the query plan. Simple scans are suitable; joins, ordering, and aggregation can prevent partitioning. See the partitionability rules.

Descriptors contain SQL and transaction identifiers and are unauthenticated. Only read descriptors from trusted sources: they execute using the receiving connection's credentials.

Supported platforms

pip selects a wheel matching the OS and architecture:

Building for an older OS requires a compatible native driver and Python dependencies. For local builds and releases, see CONTRIBUTING.md.