UQA Engine

UQA Engine is an embeddable database engine that lets one application use a PostgreSQL 18-compatible SQL surface across the behavior covered by its differential suite, full-text search, vector search, graph queries, and ranked retrieval through a shared Rust runtime.

It is designed for applications that need more than a relational table but do not want to assemble a separate database, search server, vector store, and graph engine for every query path.

What you can build

• Run relational queries, joins, aggregates, CTEs, windows, JSON operations, and transactions with PostgreSQL 18 compatibility across the verified SQL surface.
• Search text with BM25 or Bayesian BM25, retrieve vectors with KNN, and combine both signals in hybrid queries.
• Store named graphs, execute Cypher and regular path queries, and call graph traversal or centrality functions from SQL.
• Start in memory for experiments, then choose the default SQLite backend or the pure-Rust redb backend without changing the query API.
• Use the same SQL result and parameter shapes against a local or Cloud UQA node through authenticated Rust, Python, Node.js, and browser HTTP engines.
• Embed the engine in Rust or use the Python, Node.js, and browser WASM bindings included in the workspace.

New in 0.5.4

Version 0.5.4 reduces repeated catalog and index work during session creation, UPDATE and DDL. Typed constraint expressions preserve PostgreSQL identities and output; constraint catalog scans and private views retain their transaction semantics across concurrent DDL. Native Arrow and DuckDB foreign handlers are independent, opt-in features, with memory available by default. See the release history.

Catalog lookups reuse retained metadata, native SQLite equality probes seek stored keys directly, and retrieval filters read selected candidates. HNSW, IVF and full-text occurrence commits avoid repeated decoding and spill work within their existing memory allowances. Idle maintenance skips unnecessary wakeups and statistics passes.

The engine provides native DiskANN vector indexes through memory, native SQLite, SQLite Key/Value and redb. Bounded graph navigation and product quantization select candidates; complete-tensor reranking preserves canonical cosine scores and the existing probability conversion. Indexes retain transaction, rollback and reopen behavior, and EXPLAIN distinguishes estimated work from actual query counters. See the SQL configuration and score contract and matching Rust, Python, Node.js and browser examples.

Independent SQL notification subscriptions retain their original database and selected role, with bounded queues and explicit cleanup. Rust, Python, Node.js and Browser WASM also provide authenticated HTTP/SSE clients with visible loss and reconnection events for compatible servers. See the direct Rust API, language bindings and HTTP contract.

Persistent databases upgrade to SQLite record format 62 and redb record format 58; native mapping 17 and coordination format 2 are unchanged. Initial open creates physical constraint catalog records and restores retained expression identities atomically, preserving user rows. Close every owner before taking a pre-upgrade backup, update all owners together, and follow the 0.5.4 upgrade guide for migration, feature selection, custom Rust adapters and rollback requirements.

Mathematical foundation

A Typed Carrier Algebra for Unified Query Execution states the implementation-grounded theory behind UQA Engine. It distinguishes document support, weighted relations, decorated postings, ranked views, SQL bags, join tuples, graph context, and aggregate state while showing how they compose through one typed planning and execution framework.

The manuscript consolidates and revises the published work on unified query algebra, its graph-data extension, and the Bayesian framework for hybrid search. For academic use, cite the software and the papers relevant to the features used; machine-readable metadata is provided in CITATION.cff.

Try it in a terminal

Install the prebuilt Python package to get both the Python binding and the usql command:

python -m pip install uqa==0.5.4
usql

To build from this repository, you need Rust 1.90 or newer and the native build tools required by Cargo dependencies.

Start the interactive usql shell from the repository:

cargo run -p uqa-cli --bin usql

Create a table and run a text search:

CREATE TABLE notes (
    id INTEGER PRIMARY KEY,
    title TEXT,
    body TEXT
);

CREATE INDEX notes_body_gin ON notes USING gin (body);

INSERT INTO notes (id, title, body) VALUES
    (1, 'Rust async', 'Futures and the Tokio runtime'),
    (2, 'Embedded Rust', 'Drivers for constrained devices'),
    (3, 'Web application', 'Forms, routing, and templates');

SELECT id, title, _score
FROM notes
WHERE text_match(body, 'rust async')
ORDER BY _score DESC
LIMIT 5;

Use a file-backed database by adding --db:

cargo run -p uqa-cli --bin usql -- --db notes.uqa

Execute one command without entering the shell:

cargo run -p uqa-cli --bin usql -- -c "SELECT 1 AS ready"

Embed it in Rust

Add the released package to your application:

cargo add uqa@0.5.4

Korean and Japanese text analysis are separate optional features that this command does not enable: nori adds the Korean analyzer with its embedded dictionary, and kuromoji the Japanese one. Enable the ones an application uses, for example cargo add uqa@0.5.4 --features nori,kuromoji or features = ["nori", "kuromoji"] on the dependency in Cargo.toml; uqa-engine takes the same features. A build without them rejects requests for those analyzers. The usql CLI and the Python, Node.js, and browser WASM packages enable both by default.

uqa is the primary Rust package on crates.io. It is a thin facade over uqa-engine that also re-exports the core Value type; applications that need the implementation package directly can depend on uqa-engine. Public component crates including uqa-engine, uqa-client, uqa-api, and uqa-cli are also published independently. The following example creates an in-memory engine, inserts data, and runs SQL through the same interface used by a persistent engine.

use uqa::Engine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Engine::new();

    engine.sql(
        "CREATE TABLE notes (id INTEGER PRIMARY KEY, title TEXT, body TEXT)",
        &[],
    )?;
    engine.sql(
        "CREATE INDEX notes_body_gin ON notes USING gin (body)",
        &[],
    )?;
    engine.sql(
        "INSERT INTO notes (id, title, body) VALUES
         (1, 'Rust async', 'Futures and the Tokio runtime'),
         (2, 'Embedded Rust', 'Drivers for constrained devices')",
        &[],
    )?;

    let result = engine.sql(
        "SELECT id, title, _score
         FROM notes
         WHERE text_match(body, 'rust async')
         ORDER BY _score DESC",
        &[],
    )?;

    for row in result.rows {
        println!("{row:?}");
    }

    Ok(())
}

Run the complete example from this repository:

cargo run -p uqa-engine --example text_search

Additional runnable examples cover hybrid search and encrypted storage:

cargo run -p uqa-engine --example hybrid_search
cargo run -p uqa-engine --example sqlcipher_encrypted_catalog
cargo run -p uqa-engine --example compressed_encrypted_catalog

Connect to a local or Cloud UQA node

uqa-client::HttpEngine sends SQL directly to the authenticated HTTP data plane shared by local and Cloud nodes. Native applications can resolve a project through the installed CLI once during construction, while services can continue to supply an explicit URL and token or trusted UQA_URL and UQA_TOKEN environment variables.

use uqa_client::{HttpEngine, SQLParam};
use uqa_core::Value;

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let engine = HttpEngine::cloud("notes", Some("example")).await?;
let result = engine
    .sql(
        "SELECT id, title FROM notes WHERE id = $1",
        &[SQLParam::scalar(Value::Int(42))],
    )
    .await?;
assert_eq!(result.rows.len(), 1);
# Ok(())
# }

Python and Node.js provide matching local and cloud project constructors; browsers retain explicit URL/token and environment construction because they cannot execute the CLI or access its credential store. Every client calls /v1/sql, /v1/sql/batch, and /v1/sql/stream directly after construction. See the HTTP Engine reference for connection, binding examples, result, streaming, CORS, and security contracts.

Choose a query path

Persistence and encryption

Engine::new() keeps data in memory, while Engine::open(path) and usql --db <path> use the default persistent SQLite backend. Persistent engines restore schemas, documents, text postings, graphs, scoring parameters, models, views, and statistics when reopened.

uqa-storage owns provider-independent contracts and shared data structures, while uqa-storage-sqlite owns SQLite connections, catalogs, indexes, transactions, graph persistence, and compressed storage. Rust callers using concrete storage types must use the updated provider imports.

Applications that want a pure-Rust single-file store can compose the engine with uqa-storage-redb. The provider owns the database, and every Engine::new_session() receives independent transaction state over the same file.

use std::sync::Arc;
use uqa::Engine;
use uqa_storage::PersistentStorageProvider;
use uqa_storage_redb::RedbStorage;

let provider: Arc<dyn PersistentStorageProvider> =
    Arc::new(RedbStorage::open("notes.redb")?);
let engine = Engine::from_persistent_provider(provider)?;
# Ok::<(), Box<dyn std::error::Error>>(())

The redb path supports the catalog, documents, full-text search, graphs, durable B-tree indexes, exact brute-force vectors, physical IVF and HNSW indexes, transactions, and savepoints. It uses the same SQL DDL and query API as SQLite; the main capability difference is that redb does not provide encryption at rest. See the Key/Value storage design for storage and transaction details.

Persistent full-text indexes use clustered postings rather than one physical row or key per (term, doc_id): one (table, field, term, doc_id / 65,536) value stores delta-encoded document IDs, term frequencies, and document lengths, while positions live in a separate value. Ranking opens score-only cursors, reuses one decode buffer for at most 128 postings per block, and leaves positional payloads unread unless a positional consumer asks for them. SQLite schema v22 and the shared Key/Value backend automatically migrate the previous per-document posting format on open; each SQLite or redb rewrite is atomic, idempotent, and rolls back without changing the old data when validation fails.

Security-sensitive deployments should use the SQLCipher path exposed by Engine::open_encrypted. Compressed encrypted containers are also available when compression is required, but they have a narrower, explicitly documented threat model and require an external trusted anchor for whole-file rollback detection.

Read the compressed VFS security contract before selecting that format.

Language bindings

Released Node.js applications install @cognica-io/uqa from npm; npm selects an exact-version native optional package under @cognica-io for the current supported platform. Browser applications install the independent @cognica-io/uqa-wasm package.

Prebuilt Linux Python wheels target glibc 2.28 or newer.

The memory foreign-data wrapper is included by default. Enable arrow for native Arrow IPC sources or duckdb for native DuckDB sources; the features are independent and disabled by default. Foreign-table definitions and catalog inspection remain available for every handler; see the feature contract.

Build and test

Install the commit hook, then build and test the workspace:

bash scripts/install-git-hooks.sh
cargo build --workspace --locked
cargo test --workspace --locked

Run a focused test during development:

cargo test -p uqa-engine --test integration queries::sql_joins::

See CONTRIBUTING.md for contributor checks, crate ownership for implementation boundaries, and verification for compatibility tests and benchmarks.

PostgreSQL 18 compatibility

Compatibility remains incomplete. The manual and manifest-derived ledger distinguish verified behavior from remaining SQL, type, protocol and client work. The unmodified upstream regression corpus has PostgreSQL reference evidence; its complete UQA execution is still unaudited.

The repository includes a deterministic TPC-H-derived scale-factor 0.001 fixture with all 22 default queries. The self-contained correctness gate compares exact columns, row order, NULLs, text bytes, and type-aware canonical numeric values with checked-in PostgreSQL 18.4 results:

cargo test -p uqa-engine --test integration sql_tpch::

The broader PostgreSQL 18.4 gate validates the compatibility manifest and compares every checked-in value or SQLSTATE probe with a live server. Build usql in release mode before running it:

cargo build --release -p uqa-cli
python3 tests/parity/pg18/run_diff.py --validate-manifest
python3 tests/parity/pg18/run_diff.py

Stateful routine, constraint, type-and-temporal, trigger, and rewrite-rule oracles plus the pinned psycopg, pgx, and node-postgres matrix are documented in PG18 differential probes. The current milestone and open-gate ledger is the PostgreSQL 18 compatibility plan.

Current performance verification uses the deterministic regression inventory and the separate controlled-runner protocol. Dated TPC-H, posting and relevance measurements remain in the TPC-H benchmark and performance design; they are not current-release measurements.

Contributor checks, benchmark build gates, and repository conventions are documented in CONTRIBUTING.md.

Documentation

Project layout

Cypher default-label validation belongs to uqa-graph; Engine supplies the selected graph snapshot and retains its transaction boundary. Graph tests cover nested pattern requirements and missing-label diagnostics, while durable graph lifecycle tests remain with Engine.

RPQ syntax and its parser are shared through uqa-core; planner estimates no longer import the graph runtime. Existing graph syntax imports remain available, and parser tests move with their implementation.

The repository is a Rust workspace with small crates for the algebra, storage, scoring, graph, SQL, planning, execution, engine, CLI, APIs, and language bindings. The full dependency map and ownership rules live in the system architecture, keeping this README focused on using the project.

Model training uses shared native execution for Rust and SQL callers: uqa-execution converts projected table rows, parses training JSON, invokes uqa-ml, and publishes the trained result through Engine's model transaction boundary. Pure training-input and persisted IVF-parameter tests live with their owning crates. See the ownership design.

Creation namespace selection and index-target visibility run in SQL and native execution. Engine lends live schema, role, relation, and session guards; CTAS and index creation retain their existing authorization and collision-check order.

Contributing

See CONTRIBUTING.md for local gates, test conventions, crate boundaries, pull request guidelines, and the current contributor-licensing requirement.

AI-assisted contributions follow AI_POLICY.md, including semantic preservation, algebraic proofs for feature additions, and maintainer judgment on code and design.

License

UQA Engine is open-source software licensed under AGPL-3.0-only. See LICENSE.

Two optional additional permissions are available:

• the FOSS exception lets a complete qualifying open-source application retain its OSI-approved license while UQA Engine and modifications to UQA Engine remain under the AGPL; and
• the noncommercial application exception lets a qualifying personal, educational, academic, or charitable application keep its independent code under terms chosen by its author while UQA Engine and modifications to UQA Engine remain under the AGPL.

Separate commercial licensing is available for proprietary applications, closed modifications, SaaS, and OEM distribution. The complete decision guide is in the licensing policy.