apexbase 1.38.0

ApexBase is a high-performance embedded HTAP database with a Rust core and a Python-first API.

Install it, choose persistent .apex files or a true process-local in-memory database, run analytical SQL, import/export DataFrames, and optionally expose persistent data through PostgreSQL Wire or Arrow Flight. No separate database service is required.

Why ApexBase

Install

pip install apexbase

Build from source:

python -m pip install maturin
maturin develop --release

30-Second Example: FTS + SQL + Vector Search In One Local File

from apexbase import ApexClient

with ApexClient("./rag-data") as client:
    client.execute("""
        CREATE TABLE articles (
            title TEXT,
            body TEXT,
            category TEXT,
            views INT,
            embedding FLOAT16_VECTOR
        )
    """)
    client.use_table("articles")

    client.store([
        {
            "title": "Rust-powered local analytics",
            "body": "A columnar embedded database for fast SQL and search.",
            "category": "database",
            "views": 4200,
            "embedding": [0.10, 0.82, 0.20],
        },
        {
            "title": "Hybrid retrieval for RAG",
            "body": "Combine full-text recall, SQL filters, and semantic vector ranking.",
            "category": "ai",
            "views": 6100,
            "embedding": [0.16, 0.74, 0.58],
        },
        {
            "title": "SQLite migration notes",
            "body": "Move local applications to an analytical embedded store.",
            "category": "database",
            "views": 2600,
            "embedding": [0.80, 0.12, 0.10],
        },
    ])

    client.execute("CREATE FTS INDEX ON articles(title, body)")

    # FTS recall + structured SQL guardrails + pgvector-style semantic rerank.
    df = client.execute("""
        SELECT
            title,
            category,
            views,
            cosine_distance(embedding, [0.12, 0.78, 0.25]) AS semantic_dist
        FROM articles
        WHERE MATCH('database')
          AND category = 'database'
          AND views > 3000
        ORDER BY semantic_dist
        LIMIT 5
    """).to_pandas()

    print(df)

ApexBase gives you pgvector-style semantic search, SQL filters, and full-text search in the same embedded database file. It is the kind of stack you would otherwise assemble from SQLite/DuckDB + FTS + pgvector, but without a server process or a separate search/vector service; results still convert directly to Pandas, Polars, or Arrow.

For scratch work, tests, and short-lived analytics, use the same API without creating database, catalog, WAL, delta, or index files:

from apexbase import ApexClient

with ApexClient(":memory:") as client:
    client.execute("CREATE TABLE metrics (name TEXT, value DOUBLE)")
    client.execute("INSERT INTO metrics VALUES ('latency_ms', 2.4)")
    print(client.execute("SELECT AVG(value) AS average FROM metrics").scalar())

For the smallest SQL-only scripts, apexbase.execute(...) uses one lazily created process-local in-memory connection shared by later module-level calls; create a table before running table-scoped queries.

Performance At A Glance

The latest retained complete public snapshot uses 1,000,000 tabular rows and 1,000,000 vectors x 128 dimensions on Apple arm64 with Python 3.12.

Benchmarks are workload-sensitive. The default benchmark command tracks this public scoreboard; extended diagnostics live in benchmarks/bench_vs_sqlite_duckdb_extended.py. See the full reproducible setup in the Performance documentation.

Starting with 1.33, filtered grouped queries can use a shared physical scan protocol across persisted base data and delta/overlay state. The protocol keeps predicate selection, grouping, HAVING, and ordered TopK as composable operators with a generic fallback for unsupported or inexact predicate forms. v1.34.0 adds a serial row-group pipeline, bounded parallel scan/fold execution, streaming overlay reads, and a per-query aggregation memory budget on top of it. See Scan & Physical Execution and Resource Ownership.

v1.37.0 and v1.38.0 are the V5 architecture preview releases. Files may declare format version 5 and carry V5-only column encodings, _id ranges are reserved against a shared on-disk high-water mark so several processes can write one table, and new read lanes cover direct point reads, executor index lookups, full-text row-selection pushdown, and distance TopK. v1.38.0 keeps the same format and fixes the SELECT t.* … JOIN projection failure, the SQL point-lookup latency gap and the read degradation a predicate UPDATE used to leave behind. V5 builds read V4 files; to keep a file readable by a pre-V5 build, set APEX_WIDTH_FIDELITY=0 to write the V4 layout. The format may still gain encodings before V5 is declared stable, so treat it as a preview and see the release notes and Storage Architecture for the compatibility rules.

Documentation

Start here: https://birchkwok.github.io/apexbase/

Interfaces

# Embedded Python
python -c "from apexbase import ApexClient; print(ApexClient)"

# PostgreSQL Wire + Arrow Flight together
apexbase-serve --dir ./data

# Individual protocol servers
apexbase-server --dir ./data --port 5432
apexbase-flight --dir ./data --port 50051

Lance Interop

from apexbase import ApexClient

with ApexClient("./data") as client:
    client.use_table("articles")
    client.to_lance("./articles.lance")

with ApexClient("./imported") as client:
    client.from_lance("./articles.lance", table_name="articles")

Lance conversion uses Arrow tables as the handoff path. This keeps the in-process conversion lean and Arrow-native, while each format still writes its own on-disk layout.

License

Apache-2.0