UQA Engine

UQA Engine is an embeddable database engine that lets one application use a PostgreSQL 18-compatible SQL surface across the behavior covered by its differential suite, full-text search, vector search, graph queries, and ranked retrieval through a shared Rust runtime.

It is designed for applications that need more than a relational table but do not want to assemble a separate database, search server, vector store, and graph engine for every query path.

What you can build

• Run relational queries, joins, aggregates, CTEs, windows, JSON operations, and transactions with PostgreSQL 18 compatibility across the verified SQL surface.
• Search text with BM25 or Bayesian BM25, retrieve vectors with KNN, and combine both signals in hybrid queries.
• Store named graphs, execute Cypher and regular path queries, and call graph traversal or centrality functions from SQL.
• Start in memory for experiments, then choose the default SQLite backend or the pure-Rust redb backend without changing the query API.
• Use the same SQL result and parameter shapes against a local or Cloud UQA node through authenticated Rust, Python, Node.js, and browser HTTP engines.
• Embed the engine in Rust or use the Python, Node.js, and browser WASM bindings included in the workspace.

New in 0.4.8

Version 0.4.8 improves embedded SQL, vector and graph reads and reduces repeated work during writes. It adds B-tree index-only reads and corrects PostgreSQL 18 identity columns, integer and BYTEA input, row-trigger behavior and constraint diagnostics. See the release history.

The engine provides native DiskANN vector indexes through memory, native SQLite, SQLite Key/Value and redb. Bounded graph navigation and product quantization select candidates; complete-tensor reranking preserves canonical cosine scores and the existing probability conversion. Indexes retain transaction, rollback and reopen behavior, and EXPLAIN distinguishes estimated work from actual query counters. See the SQL configuration and score contract and matching Rust, Python, Node.js and browser examples.

Independent SQL notification subscriptions retain their original database and selected role, with bounded queues and explicit cleanup. Rust, Python, Node.js and Browser WASM also provide authenticated HTTP/SSE clients with visible loss and reconnection events for compatible servers. See the direct Rust API, language bindings and HTTP contract.

The engine also provides PostgreSQL-compatible pg_get_userbyid(oid) and PostgreSQL diagnostics for non-immutable generated expressions. Persistent databases upgrade to SQLite record format 55, native mapping 13 or redb record format 53; earlier incompatible binaries cannot reopen upgraded state. Close every database owner before taking a pre-upgrade backup, update all owners together, and follow the 0.4.8 upgrade guide. See the release history for the complete changes.

Mathematical foundation

A Typed Carrier Algebra for Unified Query Execution states the implementation-grounded theory behind UQA Engine. It distinguishes document support, weighted relations, decorated postings, ranked views, SQL bags, join tuples, graph context, and aggregate state while showing how they compose through one typed planning and execution framework.

The manuscript consolidates and revises the published work on unified query algebra, its graph-data extension, and the Bayesian framework for hybrid search. For academic use, cite the software and the papers relevant to the features used; machine-readable metadata is provided in CITATION.cff.

Try it in a terminal

Install the prebuilt Python package to get both the Python binding and the usql command:

python -m pip install uqa==0.4.8
usql

To build from this repository, you need Rust 1.90 or newer and the native build tools required by Cargo dependencies.

Start the interactive usql shell from the repository:

cargo run -p uqa-cli --bin usql

Create a table and run a text search:

CREATE TABLE notes (
    id INTEGER PRIMARY KEY,
    title TEXT,
    body TEXT
);

CREATE INDEX notes_body_gin ON notes USING gin (body);

INSERT INTO notes (id, title, body) VALUES
    (1, 'Rust async', 'Futures and the Tokio runtime'),
    (2, 'Embedded Rust', 'Drivers for constrained devices'),
    (3, 'Web application', 'Forms, routing, and templates');

SELECT id, title, _score
FROM notes
WHERE text_match(body, 'rust async')
ORDER BY _score DESC
LIMIT 5;

Use a file-backed database by adding --db:

cargo run -p uqa-cli --bin usql -- --db notes.uqa

Execute one command without entering the shell:

cargo run -p uqa-cli --bin usql -- -c "SELECT 1 AS ready"

Embed it in Rust

Add the released package to your application:

cargo add uqa@0.4.8

Korean and Japanese text analysis are separate optional features that this command does not enable: nori adds the Korean analyzer with its embedded dictionary, and kuromoji the Japanese one. Enable the ones an application uses, for example cargo add uqa@0.4.8 --features nori,kuromoji or features = ["nori", "kuromoji"] on the dependency in Cargo.toml; uqa-engine takes the same features. A build without them rejects requests for those analyzers. The usql CLI and the Python, Node.js, and browser WASM packages enable both by default.

uqa is the primary Rust package on crates.io. It is a thin facade over uqa-engine that also re-exports the core Value type; applications that need the implementation package directly can depend on uqa-engine. Public component crates including uqa-engine, uqa-client, uqa-api, and uqa-cli are also published independently. The following example creates an in-memory engine, inserts data, and runs SQL through the same interface used by a persistent engine.

use uqa::Engine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Engine::new();

    engine.sql(
        "CREATE TABLE notes (id INTEGER PRIMARY KEY, title TEXT, body TEXT)",
        &[],
    )?;
    engine.sql(
        "CREATE INDEX notes_body_gin ON notes USING gin (body)",
        &[],
    )?;
    engine.sql(
        "INSERT INTO notes (id, title, body) VALUES
         (1, 'Rust async', 'Futures and the Tokio runtime'),
         (2, 'Embedded Rust', 'Drivers for constrained devices')",
        &[],
    )?;

    let result = engine.sql(
        "SELECT id, title, _score
         FROM notes
         WHERE text_match(body, 'rust async')
         ORDER BY _score DESC",
        &[],
    )?;

    for row in result.rows {
        println!("{row:?}");
    }

    Ok(())
}

Run the complete example from this repository:

cargo run -p uqa-engine --example text_search

Additional runnable examples cover hybrid search and encrypted storage:

cargo run -p uqa-engine --example hybrid_search
cargo run -p uqa-engine --example sqlcipher_encrypted_catalog
cargo run -p uqa-engine --example compressed_encrypted_catalog

Connect to a local or Cloud UQA node

uqa-client::HttpEngine sends SQL directly to the authenticated HTTP data plane shared by local and Cloud nodes. Native applications can resolve a project through the installed CLI once during construction, while services can continue to supply an explicit URL and token or trusted UQA_URL and UQA_TOKEN environment variables.

use uqa_client::{HttpEngine, SQLParam};
use uqa_core::Value;

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
let engine = HttpEngine::cloud("notes", Some("example")).await?;
let result = engine
    .sql(
        "SELECT id, title FROM notes WHERE id = $1",
        &[SQLParam::scalar(Value::Int(42))],
    )
    .await?;
assert_eq!(result.rows.len(), 1);
# Ok(())
# }

Python and Node.js provide matching local and cloud project constructors; browsers retain explicit URL/token and environment construction because they cannot execute the CLI or access its credential store. Every client calls /v1/sql, /v1/sql/batch, and /v1/sql/stream directly after construction. See the HTTP Engine reference for connection, binding examples, result, streaming, CORS, and security contracts.

Choose a query path

Persistence and encryption

Engine::new() keeps data in memory, while Engine::open(path) and usql --db <path> use the default persistent SQLite backend. Persistent engines restore schemas, documents, text postings, graphs, scoring parameters, models, views, and statistics when reopened.

uqa-storage owns provider-independent contracts and shared data structures, while uqa-storage-sqlite owns SQLite connections, catalogs, indexes, transactions, graph persistence, and compressed storage. Rust callers using concrete storage types must use the updated provider imports.

Applications that want a pure-Rust single-file store can compose the engine with uqa-storage-redb. The provider owns the database, and every Engine::new_session() receives independent transaction state over the same file.

use std::sync::Arc;
use uqa::Engine;
use uqa_storage::PersistentStorageProvider;
use uqa_storage_redb::RedbStorage;

let provider: Arc<dyn PersistentStorageProvider> =
    Arc::new(RedbStorage::open("notes.redb")?);
let engine = Engine::from_persistent_provider(provider)?;
# Ok::<(), Box<dyn std::error::Error>>(())

The redb path supports the catalog, documents, full-text search, graphs, durable B-tree indexes, exact brute-force vectors, physical IVF and HNSW indexes, transactions, and savepoints. It uses the same SQL DDL and query API as SQLite; the main capability difference is that redb does not provide encryption at rest. See the Key/Value storage design for storage and transaction details.

Persistent full-text indexes use clustered postings rather than one physical row or key per (term, doc_id): one (table, field, term, doc_id / 65,536) value stores delta-encoded document IDs, term frequencies, and document lengths, while positions live in a separate value. Ranking opens score-only cursors, reuses one decode buffer for at most 128 postings per block, and leaves positional payloads unread unless a positional consumer asks for them. SQLite schema v22 and the shared Key/Value backend automatically migrate the previous per-document posting format on open; each SQLite or redb rewrite is atomic, idempotent, and rolls back without changing the old data when validation fails.

Security-sensitive deployments should use the SQLCipher path exposed by Engine::open_encrypted. Compressed encrypted containers are also available when compression is required, but they have a narrower, explicitly documented threat model and require an external trusted anchor for whole-file rollback detection.

Read the compressed VFS security contract before selecting that format.

Language bindings

Released Node.js applications install @cognica-io/uqa from npm; npm selects an exact-version native optional package under @cognica-io for the current supported platform. Browser applications install the independent @cognica-io/uqa-wasm package.

Prebuilt Linux Python wheels target glibc 2.28 or newer because the bundled DuckDB runtime requires the modern C++11 ABI.

Build and test

Install the versioned commit hook once per clone:

bash scripts/install-git-hooks.sh

SQL owns executable statement schemas and mutation parameter analysis, TRUNCATE target expansion and foreign-key ordering, view namespaces, persisted metadata validation, source binding and dependency analysis, replacement row types and ownership rules, transaction-block diagnostics, prepared declarations, argument validation and result descriptor identity, mutation and declaration analysis, relation lookup candidate ordering, foreign-table schema envelopes, sequence binding, literal and regclass dependency analysis, and declaration rewrites, trigger/rule definitions, lookup semantics, dependency rewrites, durable envelopes and partition-name checks, stored relation and routine binding, overload and CALL rules, stored-object dependencies, routine replacement and privileges, table/column GRANT and REVOKE candidates, view access rules, schema authorization binding, privilege inquiries, and graph and scalar-projection argument rules. Retrieval predicates, FTS and operator joins lower into a declarative IR without runtime scorers, fusers, closures or physical index choices. Shared agtype and retrieval configuration values live in core.

The planner schedules statement analysis before optimizer input capture and constant evaluation, and owns source and rewrite-rule pruning, statement statistics, prepared-plan costs and custom/generic selection, parameter specialization, retrieval optimizer setup, index and graph statistics, independent relation and operator-join costs, indexed access selection and physical text top-k planning. Execution owns portal declaration and directional row streaming, batch and cursor scheduling, statement cache use, snapshot-bound re-lowering, transaction cleanup and result delivery, exhaustive unified statement dispatch, cancellation and CTE validation before transaction checks, nested EXPLAIN and prepared execution, view creation, refresh, restoration, deletion and reference publication, including migration rollback, statement read-only validation and error rollback, TRUNCATE authorization and trigger scheduling, prepared-argument evaluation and conversion order, scoped scalar-subquery type binding, scalar-subquery caches and correlated EXISTS execution, mutation loops, trigger/rule registry execution and restoration, foreign-table creation and removal, pinned/live foreign lookup, durable sequence restoration, sequence value allocation and session-cache scheduling, ordinary-table DROP dependency and cascade publication, sequence-owner enumeration, implicit-owner migration and attachment, sequence DROP scheduling and ordered dependency removal, definition persistence, identity migration and server lifecycle, table-shaped privilege persistence and publication, table-owner transfer with owned-sequence ACL publication, retained-table and foreign-table authority, maintenance target authorization, role, schema and routine lifecycle scheduling, catalog publication, routine invocation and caller restoration, table-function streams, scalar dispatch, score provenance and highlight rendering. It also owns retrieval row dispatch, bounded vector-pool interpretation, physical fusion construction, tree execution, cross-relation joins, score composition and model inference.

Engine supplies retained catalog guards, table generations, storage operations, cache state and statement/transaction boundaries through typed adapters. Its query API owns statement gates and clocks; document reads live with table storage. Compiled-statement execution and catalog-bound planning order belong to execution. Engine has no SQL implementation directory; its public sql namespace re-exports native result types and formatting. Native batch and cursor scheduling invoke the same execution dispatcher, retaining each row-lock guard through its original cleanup boundary. SQL owns cursor declaration restrictions, relation and sequence binding, and table/graph dependency analysis; native portal execution captures its query scope after the first row request, while Engine retains the child session, pinned snapshots and delegated statement gate. Prepared descriptors and cache entries are SQL-owned; execution retains the registration write guard while constructing metadata, and the planner selects custom/generic variants before the session publishes usage against the captured plan identity. The planner implements a SQL-plan optimization contract; execution does not depend on the planner. Public retrieval APIs retain their existing entry points. Every commit checks the staged dependency graph, including build and platform-specific edges and transitive ownership boundaries. See crate ownership, planning and execution, and the SQL boundary design.

Build the complete workspace:

cargo build --workspace --locked

Run the test suite:

cargo test --workspace --locked

Run the optimized persistent text top-k engine benchmark; cargo bench builds and executes the release benchmark profile, this target uses a real SQLite file rather than the in-memory engine, and it invokes Engine::search_profiled directly so cursor and ranking costs remain isolated from SQL planning and row projection:

cargo bench -p uqa-engine --bench text_top_k --locked -- --warm-up-time 2 --measurement-time 5 --sample-size 30 --noplot

Run the optimized persistent SQL vector-search performance and exact-ground-truth quality benchmark. The default profile loads 100,000 128-dimensional vectors into a real SQLite file, reopens it for each exact, IVF, and HNSW phase, and drives every query through Engine::sql:

bash scripts/run-vector-search-benchmark.sh

Run the real-data BEIR hybrid-search benchmark after installing its pinned Python embedding dependency. The runner downloads and verifies SciFact, generates MiniLM embeddings, loads a persistent SQLite file through SQL, creates GIN and HNSW indexes through SQL, reopens it, and executes all BM25, vector, and hybrid queries through Engine::sql:

python3 -m pip install -r benchmarks/beir/requirements.txt
bash scripts/run-beir-benchmark.sh

The combined report includes exact, IVF, and HNSW SQL query latency and throughput, SQL load and index-construction throughput, recall@10, top-1 accuracy, MRR@10, exact top-k set rate, and cosine-score error. Pass smoke or large to select the 10,000-row or 1,000,000-row profile; the deterministic workload, measured boundary, metric definitions, quality floors, output files, and limitations are documented in the vector-search benchmark.

Integration tests are consolidated into a small set of domain harnesses so a workspace test does not pay one linker and process-startup cost per source file. Individual modules remain directly selectable during development:

cargo test -p uqa-engine --test integration queries::sql_joins::
cargo test -p uqa-sql --test integration parser_fuzz::

PostgreSQL 18 compatibility

The repository includes a deterministic TPC-H-derived scale-factor 0.001 fixture with all 22 default queries. The self-contained correctness gate compares exact columns, row order, NULLs, text bytes, and type-aware canonical numeric values with checked-in PostgreSQL 18.4 results:

cargo test -p uqa-engine --test integration sql_tpch::

The broader PostgreSQL 18.4 gate validates the compatibility manifest and compares every checked-in value or SQLSTATE probe with a live server. Build usql in release mode before running it:

cargo build --release -p uqa-cli
python3 tests/parity/pg18/run_diff.py --validate-manifest
python3 tests/parity/pg18/run_diff.py

Stateful routine, constraint, type-and-temporal, trigger, and rewrite-rule oracles plus the pinned psycopg, pgx, and node-postgres matrix are documented in PG18 differential probes. The current milestone and open-gate ledger is the PostgreSQL 18 compatibility plan.

Release-mode timing uses a machine-readable runner rather than test-profile execution:

cargo build --release -p uqa-engine --example tpch_runner --locked
target/release/examples/tpch_runner --iterations 201

In the 2026-08-09 local arm64 development snapshot, UQA matched all 22 results and had a lower median latency than PostgreSQL 17 on 14 of 22 queries. This is a small developer-machine compatibility workload, not a compliant or audited TPC-H result. The complete fixture provenance, per-query measurements, and reproduction rules are in the TPC-H compatibility benchmark; the broader benchmark methodology is in the performance design document.

The 2026-08-11 clustered-posting pass measured release-profile persisted Block-Max WAND at 1.0142 ms and WAND at 0.9337 ms on the direct 5,000-document reopened-SQLite probe, down 73.7% and 76.5% from the preceding 3.8584 ms and 3.9801 ms baselines. The 2026-08-12 pinned SciFact run separately measured the current exact hybrid_log_odds contract at 0.7226 NDCG@10, 0.6820 MAP@10, 0.8322 Recall@10, and 3.29 ms per query; it passed every absolute and comparative gate. Commands, measured boundaries, validity rules, complete tables, and limitations are recorded in the performance design document.

Contributor checks, benchmark build gates, and repository conventions are documented in CONTRIBUTING.md.

Documentation

Project layout

Cypher default-label validation belongs to uqa-graph; Engine supplies the selected graph snapshot and retains its transaction boundary. Graph tests cover nested pattern requirements and missing-label diagnostics, while durable graph lifecycle tests remain with Engine.

RPQ syntax and its parser are shared through uqa-core; planner estimates no longer import the graph runtime. Existing graph syntax imports remain available, and parser tests move with their implementation.

The repository is a Rust workspace with small crates for the algebra, storage, scoring, graph, SQL, planning, execution, engine, CLI, APIs, and language bindings. The full dependency map and ownership rules live in the system architecture, keeping this README focused on using the project.

Model training uses shared native execution for Rust and SQL callers: uqa-execution converts projected table rows, parses training JSON, invokes uqa-ml, and publishes the trained result through Engine's model transaction boundary. Pure training-input and persisted IVF-parameter tests live with their owning crates. See the ownership design.

Creation namespace selection and index-target visibility run in SQL and native execution. Engine lends live schema, role, relation, and session guards; CTAS and index creation retain their existing authorization and collision-check order.

Contributing

See CONTRIBUTING.md for local gates, test conventions, crate boundaries, pull request guidelines, and the current contributor-licensing requirement.

AI-assisted contributions follow AI_POLICY.md, including semantic preservation, algebraic proofs for feature additions, and maintainer judgment on code and design.

License

UQA Engine is open-source software licensed under AGPL-3.0-only. See LICENSE.

Two optional additional permissions are available:

• the FOSS exception lets a complete qualifying open-source application retain its OSI-approved license while UQA Engine and modifications to UQA Engine remain under the AGPL; and
• the noncommercial application exception lets a qualifying personal, educational, academic, or charitable application keep its independent code under terms chosen by its author while UQA Engine and modifications to UQA Engine remain under the AGPL.

Separate commercial licensing is available for proprietary applications, closed modifications, SaaS, and OEM distribution. The complete decision guide is in the licensing policy.