Skip to main content

Agentic AI database. Storage engine with MVCC, WAL, LSM compaction.

Project description

uldb

The database for AI agents.

Rust Tests Python Tests License

Benchmarks | Quick Start | Architecture | API Reference


uldb is a storage engine purpose-built for agentic AI workloads. MVCC transactions, WAL crash recovery, LSM compaction, and five search indices in a single binary with zero required external services.

Agents get isolated branch workspaces with snapshot-level reads and atomic merge-or-rollback semantics. Every write is crash-safe. Every search returns ranked results across all indices via reciprocal rank fusion.

Native ulmen-core integration for typed agent payload storage.

import uldb

app = uldb.open("./my_project")

app.put("auth.py::validate", b"def validate(token): ...")
results = app.search("validate token")

with app.agent("refactor-auth") as agent:
    docs = agent.search("validate token")
    agent.put("auth.py::validate", b"def validate_v2(token): ...")
    # auto-merge on success, auto-rollback on failure

app.close()
use uldb::engine::{Engine, EngineConfig};
use uldb::agent_store;
use ulmen_core::*;

let mut engine = Engine::open(EngineConfig::new("./data")).unwrap();

// Store typed agent payloads
let payload = AgentPayload { /* ... */ };
agent_store::store_payload(&mut engine, "session:001", &payload).unwrap();

// Search across agent records
let results = agent_store::search_records(&mut engine, "authentication", 10);

Benchmarks

AMD Ryzen 7 4700U, 8 cores, NVMe SSD. Release build.

Throughput

Operation Speed Latency
PUT sequential 141K ops/sec 7.1 us
PUT batch 171K ops/sec 5.9 us
GET (hit) 4.2M ops/sec 238 ns
GET (cached) 10.0M ops/sec 100 ns
DELETE 349K ops/sec 2.9 us
SCAN 100 keys 230K ops/sec 4.4 us
BM25 query 83K queries/sec 12.1 us
Fuzzy query 3.9K queries/sec 257 us
Snapshot create 10.3M ops/sec 97 ns

Agentic Workloads

Workload Throughput
Codebase ingest (10K symbols) 143K symbols/sec
Codebase ingest (50K symbols) 126K symbols/sec
Agent session (1K mixed ops) 25.3K ops/sec
4 concurrent agents 113K ops/sec

Storage Efficiency

Metric Value
Disk overhead 1.30x raw data
WAL recovery 207K records/sec

PUT Cost Breakdown

Component Latency Share
WAL 618 ns 13%
Memtable 460 ns 10%
Index 2,543 ns 52%
HAMT 1,180 ns 25%

Installation

[dependencies]
uldb = { git = "https://github.com/makroumi/uldb" }
pip install uldb

Quick Start

Python

import uldb

app = uldb.open("./my_project")

# CRUD
app.put("auth.py::validate", b"def validate(token): ...")
doc = app.get("auth.py::validate")        # returns bytes
app.delete("auth.py::validate")

# Search (BM25 + fuzzy, RRF merged)
results = app.search("validate token", limit=10)
for doc in results:
    print(f"{doc.id}: score={doc.score:.3f}")

# Fuzzy symbol search (typo-tolerant)
results = app.search_fuzzy("getUserByID", limit=5)

# Bulk ingest
app.load({
    "auth.py::validate": b"def validate(token): ...",
    "auth.py::login": b"def login(user, pwd): ...",
})

# Snapshots
snap_id = app.snapshot("before_refactor")
app.restore("before_refactor")

# Stats
print(app.stats())

app.close()

Agent Workflow

import uldb

app = uldb.open("./codebase")
app.put("auth.py::validate", b"def validate(token): return True")

# Agent: isolated workspace, auto-merge/rollback
with app.agent("security-review") as agent:
    # Search sees committed state
    docs = agent.search("validate token")

    # Writes are isolated
    agent.put("auth.py::validate", b"def validate_v2(token): return check_jwt(token)")
    agent.put("auth.py::refresh", b"def refresh(token): return renew_jwt(token)")

    # On clean exit: all writes merge to main atomically
    # On exception: all writes are discarded

# Main now has the agent's changes
assert b"validate_v2" in app.get("auth.py::validate")

Agent Rollback

try:
    with app.agent("risky-change") as agent:
        agent.put("config.py", b"SECRET_KEY = 'HACKED'")
        raise RuntimeError("detected unsafe change")
except RuntimeError:
    pass

# Config is untouched
assert b"HACKED" not in app.get("config.py")

Multiple Agents

agent_a = app.agent("agent_alpha")
agent_b = app.agent("agent_beta")

agent_a.put("shared_key", b"from alpha")
agent_b.put("shared_key", b"from beta")

# Each sees only its own writes
agent_a.discard()
agent_b.commit()

# Beta wins
assert app.get("shared_key") == b"from beta"

Vector Search

with app.agent("embedding-agent") as agent:
    results = agent.search_vector([0.1, 0.2, 0.3, ...], limit=10)

Graph Traversal

with app.agent("graph-agent") as agent:
    results = agent.search_graph("AuthService", relation="calls", depth=3)

Rust

use uldb::engine::{Engine, EngineConfig};

let mut engine = Engine::open(EngineConfig::new("./data")).unwrap();

engine.put(b"auth.py::validate", b"def validate(): ...").unwrap();

// Read (238ns)
let value = engine.get(b"auth.py::validate");

// Batch
engine.put_batch(&[
    (b"a.py::fn1", b"code1"),
    (b"a.py::fn2", b"code2"),
]).unwrap();

// Search
use uldb::query::planner::QuerySpec;
let spec = QuerySpec { text: "validate token".into(), top_k: 10, ..Default::default() };
let hits = engine.indices.query(&spec);

// Agent payloads (via ulmen-core)
use uldb::agent_store;
use ulmen_core::*;

let payload = AgentPayload { /* ... */ };
agent_store::store_payload(&mut engine, "session:001", &payload).unwrap();
let loaded = agent_store::load_payload(&engine, "session:001").unwrap();
let records = agent_store::search_records(&mut engine, "jwt token", 10);

Server

uldb serve --port 7771 --data ./data --token mytoken

connect with ulmp Python client:

from ulmp import AsyncClient

async with AsyncClient.connect("localhost", 7771, token=b"mytoken") as db:
    ns = db.namespace("my_project")
    await ns.put("auth.py::validate", b"code")
    results = await ns.search("validate")

Architecture

Engine
  WAL            crash-safe write-ahead log
  Memtable       sorted in-memory buffer
  Compaction     tiered LSM (3 levels)
  PageStore      compressed pages on disk
  Cache          LRU read cache
  HAMT           persistent hash trie (snapshots, branches)

  IndexManager
    BM25           keyword search
    HNSW           vector nearest-neighbor
    FuzzyMatcher   typo-tolerant symbol lookup
    RelationGraph  CSR graph traversal
    BloomFilter    per-page membership test
    QueryPlanner   RRF multi-index fusion

  Transactions   MVCC (snapshot + serializable isolation)
  Namespaces     repo+commit scoped isolation
  Branches       copy-on-write via HAMT
  Snapshots      zero-cost via structural sharing

  AgentStore     ulmen-core typed payload storage
  UmpHandler     ulmp wire protocol server (46 handlers)

Agent Isolation

Each agent gets a branch (copy-on-write HAMT fork):

  • Reads: snapshot state at creation + own writes
  • Writes: invisible to main and other agents until commit
  • Commit: atomic merge to main
  • Rollback: discard everything
  • Search: reflects committed main state

Sever Handlers (46 implemented)

  • Records: put, get, delete, scan, put_batch, get_batch, range_delete
  • Query: text (BM25), fuzzy, vector (HNSW), graph
  • Transactions: begin, commit, rollback, status
  • Snapshots: create, restore, delete, list
  • Branches: create, merge, rollback, diff, list
  • Namespaces: create, open, delete, list, stat, grant
  • Admin: stats, compact, config_get, config_set, backup, restore
  • Watch: register, unwatch, window (credit-based)
  • Auth: rotate_request, rotate
  • Streaming: checkpoint, stream_resume

API Reference

Python (DB)

app = uldb.open(path)
app.put(key, value_bytes)
app.get(key) -> bytes | None
app.delete(key)
app.search(query, limit=10) -> list[Document]
app.search_fuzzy(symbol, limit=5) -> list[Document]
app.scan(prefix, limit=100) -> list[Document]
app.load(records_dict) -> int
app.delete_range(start, end) -> int
app.keys(prefix, limit) -> list[str]
app.snapshot(name) -> str
app.restore(name)
app.snapshots() -> list[str]
app.stats() -> dict
app.agent(name) -> Agent
app.close()

Python (Agent)

with app.agent(name) as agent:
    agent.put(key, value)
    agent.get(key) -> Document | None
    agent.delete(key)
    agent.search(query) -> list[Document]
    agent.search_fuzzy(symbol) -> list[Document]
    agent.search_vector(embedding) -> list[Document]
    agent.search_graph(start, relation, depth) -> list[Document]
    agent.scan(prefix) -> list[Document]
    agent.load(records_dict) -> int
    agent.commit()
    agent.discard()
    agent.checkpoint(name) -> str

Rust (Engine)

Engine::open(config) -> Engine
engine.put(key, value) -> io::Result<()>
engine.get(key) -> Option<Vec<u8>>
engine.delete(key) -> io::Result<()>
engine.scan(start, end) -> Vec<(Vec<u8>, Vec<u8>)>
engine.put_batch(entries) -> io::Result<()>
engine.flush() -> io::Result<()>
engine.close() -> io::Result<()>

Rust (Agent Store)

agent_store::store_payload(engine, key, payload) -> io::Result<()>
agent_store::load_payload(engine, key) -> Result<AgentPayload>
agent_store::search_records(engine, query, limit) -> Vec<(AgentRecord, f64)>
agent_store::append_records(engine, key, records) -> io::Result<()>
agent_store::get_records_by_type(engine, key, type) -> Vec<AgentRecord>
agent_store::get_recent_records(engine, key, limit) -> Vec<AgentRecord>
agent_store::list_sessions(engine) -> Vec<String>
agent_store::delete_payload(engine, key) -> io::Result<()>

Testing

# Rust (260 tests)
cargo test --features server

# Python (86 tests)
source .venv/bin/activate
maturin develop --release --features python
pytest tests/python/ -v

# Benchmarks
cargo bench --bench engine_bench

Ecosystem

  • ulmen - Serialization + agent protocol
  • uldb - Storage, indexing, caching
  • ulmp - Wire protocol, networking

License

Business Source License 1.1. See LICENSE.

Copyright (c) 2026 El Mehdi Makroumi.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ulmendb-0.1.0.tar.gz (170.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ulmendb-0.1.0-cp314-cp314-manylinux_2_34_x86_64.whl (541.6 kB view details)

Uploaded CPython 3.14manylinux: glibc 2.34+ x86-64

File details

Details for the file ulmendb-0.1.0.tar.gz.

File metadata

  • Download URL: ulmendb-0.1.0.tar.gz
  • Upload date:
  • Size: 170.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.13.3

File hashes

Hashes for ulmendb-0.1.0.tar.gz
Algorithm Hash digest
SHA256 aa8b39c0aabb39cd004f48eaeae204b2dd9a52c1175e04fa6215505ced25f7ba
MD5 041c39e5ed74b8420b70df00f3528df3
BLAKE2b-256 aeed82cc08d4450e364581682be97baf76bd41c74e9addbba825073d23fd0259

See more details on using hashes here.

File details

Details for the file ulmendb-0.1.0-cp314-cp314-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for ulmendb-0.1.0-cp314-cp314-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 cf3be6c0683622806652a1320dc26eeeb82e8b56ec4cf4c631252202a0477c5f
MD5 a42cc8fd7185f7a223673d3a3f643b55
BLAKE2b-256 b78731f39db44f59d17c337cbbf07d4f108a593b3ff8987bd905f33738127b4e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page