bareduckdb 0.3.142

Minimal Cython-based DuckDB Bindings

bareduckdb

Simplified, Dynamically Linked DuckDB Python Bindings — Fast, simple, and free-threaded.

---

Overview

bareduckdb provides extensible and easy to build Python bindings to DuckDB using Cython.

• Simple ~4k lines of code - easy to extend or customize
• Arrow-first data conversion supporting Polars, PyArrow, and Pandas
• Support for latest Python features Free threading, subinterpreters, and asyncio
• Dynamically linked to DuckDB's official library

Installation

From PyPI

pip install bareduckdb

From Source

git clone --recurse-submodules https://github.com/paultiq/bareduckdb.git
cd bareduckdb
uv sync -v # or: pip install -e .

Basic Usage

import bareduckdb

# Connect to in-memory database
conn = bareduckdb.connect()

# Execute query and get Arrow Table
result = conn.execute("SELECT 42 as answer").arrow_table()
print(result)

# Convert to Polars/Pandas/PyArrow
df_polars = conn.execute("SELECT * FROM range(100)").pl()
df_pandas = conn.execute("SELECT * FROM range(100)").df()

Async API

import asyncio
from bareduckdb.aio import connect_async

async def run_query():
    async with await connect_async() as conn:
        result = await conn.execute("SELECT * FROM generate_series(1, 1000)")
        return result

result = asyncio.run(run_query())

Polars Integration

import bareduckdb
import polars as pl

conn = bareduckdb.connect()

# Polars -> DuckDB (Arrow Capsule protocol)
df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
conn.register("my_table", df)

# DuckDB -> Polars (direct conversion)
result = conn.execute("SELECT * FROM my_table", output_type="polars")

---

Architecture

Design Principles

1. Keep it in Python — Business logic lives in Python, not Cython/C++
2. No GIL interaction from DuckDB threads — All Python operations happen before/after query execution
3. Semantic Versioning — Strict stability guarantees
4. Arrow-first — All data types map through Arrow's type system

Why Arrow-First?

By forcing all conversions through Arrow, bareduckdb achieves:

• Consistent type mappings across Polars/Pandas/PyArrow
• Reduced code complexity (no per-library conversion paths)
• Better memory efficiency (zero-copy where possible)
• Future-proof (Arrow is the lingua franca for columnar data)

Thread Safety & Free-Threading

Free-threading support (Python 3.13+):

• No global locks in critical paths
• DuckDB threads never acquire the GIL
• Safe concurrent query execution in --disable-gil mode
• Atomic operations for Arrow stream coordination

---

APIs

bareduckdb provides multiple API layers for different use cases:

1. Core API (bareduckdb.core)

Minimal, no-frills interface for maximum performance.

from bareduckdb.core import Connection
conn = Connection()
result = conn.execute("SELECT 1")

2. Async API (bareduckdb.aio)

Non-blocking operations with async/await.

from bareduckdb.aio import connect_async
conn = await connect_async()
result = await conn.execute("SELECT 1")

3. Compatibility API (bareduckdb.compat)

Familiar interface similar to duckdb-python (with intentional differences).

import bareduckdb
conn = bareduckdb.connect()
result = conn.sql("SELECT 1")  # Eager execution

4. DBAPI 2.0 (bareduckdb.dbapi)

Standard Python database interface for compatibility with tools like SQLAlchemy.

from bareduckdb.dbapi import connect
conn = connect()
cursor = conn.cursor()
cursor.execute("SELECT 1")

---

Key Differences

Not (Yet?) Supported

• No lazy DuckDBPyRelation objects
• No Python UDFs
• No automatic DataFrame registration (use .register() explicitly)
• Added Inline Registration: .execute("....query...", data={"name": df})
• No fsspec integration

Arrow Enhancements

<TBD: Document>

• Capsule vs Table registration
• Deadlock detection
• Cardinality & TopN
• C++ implementation of Arrow Pushdown

Type Mappings

All types convert through Arrow:

• UUIDs: Returned as strings (Arrow doesn't have native UUID type)
• Decimals: Arrow Decimal128/Decimal256
• Timestamps: Arrow Timestamp with timezone preservation
• Nested Types: Struct/List/Map fully supported

Development

Building from Source

# Clone with submodules (sparse checkout is automatic)
git clone --recurse-submodules https://github.com/paultiq/bareduckdb.git
cd bareduckdb

# Install development dependencies
uv sync

# Build in development mode
pip install -e .

* Note 1: DuckDB submodule version must match the library version. * Note 2: PyArrow version must match the runtime version for Table registration / Pushdown

Disclaimer

For official Python bindings, see: https://github.com/duckdb/duckdb-python

License

bareduckdb is licensed under the MIT License. See LICENSE for details.

All original copyrights are retained by their respective owners, including DuckDB and DuckDB-Python