trw-memory: persistent, local-first memory for AI agents

trw-memory is a persistent memory engine for AI agents: an agent memory layer that gives LLM agents long-term memory across sessions, stored locally in SQLite. Use it as an async Python SDK, a CLI, or an MCP memory server. The core install recalls with keyword search; optional extras add hybrid retrieval (BM25 + dense vectors via sqlite-vec, fused with Reciprocal Rank Fusion) and cross-encoder reranking, alongside lifecycle scoring, tiered storage, and a knowledge graph. It is the standalone memory backend of TRW Framework and works without it.

Release status: Alpha and source-available under BSL 1.1. The public API may change before 1.0; evaluate upgrades in a test environment before production rollout.

Why trw-memory · Quick start · Python API · Conversation memory · CLI · Benchmarks · mem0 comparison · MCP server · Security and network behavior · FAQ · Development

What is trw-memory?

TRW-Memory is a standalone persistent memory engine for AI agents that gives coding agents searchable, long-lived knowledge storage. It stores learnings (patterns, gotchas, architecture decisions) in SQLite with optional YAML backup, and retrieves them using hybrid search that combines keyword matching (BM25) with dense vector similarity. It also stores conversation memory: store_conversation() keeps chat turns verbatim so they can be recalled later, RAG-style, as evidence for an answer.

Designed as the storage backend for trw-mcp and TRW Framework, but usable independently by any AI agent framework that needs persistent memory with recall.

Why trw-memory

• Local-first. With the default configuration all data lives in a local SQLite store (plus an optional YAML sidecar). There is no usage tracking or content phone-home; the only network-capable surfaces are optional model downloads and opt-in remote sync. See Telemetry and network behavior.
• No generative LLM call at write time. store_conversation() stores every turn verbatim with its date and the turn it replied to. Ingest never calls a generative model (the optional local embedding model still encodes each turn); recall does the work.
• Hybrid retrieval, optional. With the retrieval extras installed: BM25 (with stemming) + dense vectors via sqlite-vec + Reciprocal Rank Fusion + a cross-encoder re-ranker. Without them, retrieval degrades gracefully to the backend's built-in keyword search.
• Works offline. TRW_OFFLINE=1 / HF_HUB_OFFLINE=1 block model downloads; local_only: true hard-blocks all remote sync and model download. Hybrid retrieval offline needs the models already in the local cache.
• MCP memory server included. trw-memory-server exposes store, recall, search, consolidate, forget and more as MCP tools over stdio, or over a per-user loopback HTTP daemon.
• Evaluating a mem0 alternative? Using mem0's open-source evaluation suite on one LOCOMO conversation (n = 152 questions per system, one run each, local llama3.1 as answerer, judge and mem0's extraction model), neither paired test detected a statistically significant accuracy difference between trw-memory and mem0 (OSS), and trw-memory called no generative LLM to ingest the conversation. That does not establish equivalence or superiority; read the numbers and caveats first.
• Source-available. BSL 1.1, alpha; the public API may change before 1.0.

Features: hybrid retrieval, knowledge graph, lifecycle, security

• MemoryClient SDK -- High-level async Python client with store/bulk_store/store_many/recall/search/search_fts/forget plus audit_learning and review_quarantined
• Hybrid Search (BM25 + vector) -- BM25 keyword matching + dense vector similarity via sqlite-vec, combined with Reciprocal Rank Fusion (RRF). Learn more
• FTS5 keyword search -- MemoryClient.search_fts() runs indexed SQLite FTS5 keyword search with BM25 ranking over content/detail/tags for pure-keyword queries that don't need hybrid ranking; degrades to an empty result when FTS5 is unavailable
• Hybrid order preservation by default -- recall preserves the hybrid BM25+dense+RRF order when enough local candidates are already available, avoiding a legacy score-scale mismatch in tier merging. To restore the legacy tier rescore for a workload, set MEMORY_RECALL_PRESERVE_HYBRID_ORDER=false.
• Tiered Storage -- Hot/warm/cold tiers for fast recall, warm-sidecar persistence, recall-time cold promotion, and explicit sweep-based archiving/purging. Architecture details
• Semantic Deduplication -- Detects and merges near-duplicate learnings using cosine similarity (0.85 threshold)
• Knowledge Graph for AI -- Tag co-occurrence and similarity edges, BFS traversal, importance boost/decay, cross-validation propagation. Docs
• Memory Consolidation -- Episodic-to-semantic consolidation via clustering with the current shipped path using heuristic/fallback summarization
• Outcome-based memory scoring -- outcome-driven utility (Q-value) scoring with EMA updates, Ebbinghaus forgetting curve applied at query time, Bayesian MACLA calibration
• Remote Sync -- Publish/fetch learnings across installations with vector clock conflict resolution and SSE live updates
• Security -- optional AES-256-GCM field encryption (off by default), PII detection with publish-time masking, memory-poisoning anomaly detection (z-score; enforcement is opt-in), RBAC, audit trail. See Security defaults
• Agent Integration -- register_tools() for agents that expose a register_tool() or tool() API, @auto_recall decorator
• Framework Integrations -- VS Code interface contract and an OpenAI-compatible adapter
• CLI -- Full command-line interface for store, recall, search, forget, consolidate, export/import
• MCP Tools -- store, recall, search, consolidate, forget, status, audit, review, wiki-lint, and an explicit code index (index/search/symbol) — served by trw-memory-server
• Dual Storage Backends -- SQLite with keyword search (primary) + YAML (backup) with one-time migration

How trw-memory fits into TRW Framework

trw-memory is the standalone memory engine for TRW (The Real Work) — a methodology layer for AI-assisted development that provides stateless agents with a persistent memory layer designed to enable self-improvement across sessions via knowledge compounding. The outcome effect of cross-session memory on coding tasks is an open empirical question; early SWE-bench single-shot runs (n≥40) produced null. See the verification docs for the current methodology and evidence posture. It works alongside trw-mcp, the MCP server that builds its tooling on this engine.

• trw-memory (this repo): Standalone AI agent memory engine with hybrid retrieval, scoring, and lifecycle
• trw-mcp: MCP server for AI coding agents — uses trw-memory as its backend

Install and quick start

# Core local engine (SQLite + built-in keyword search)
pip install trw-memory

# Recommended hybrid retrieval
pip install "trw-memory[embeddings,vectors,bm25]"

# The full retrieval stack (same as the line above, one name)
pip install "trw-memory[all]"

By default, memories are stored in .memory/ relative to the current directory. Override with MEMORY_STORAGE_PATH env var.

For source development, clone the repository and run pip install -e ".[dev]" from trw-memory/. Tested on CPython 3.10 through 3.14; see Platform and interpreter notes for SQLite engine details.

MemoryClient (recommended)

import asyncio

from trw_memory.client import MemoryClient


async def main() -> None:
    async with MemoryClient(namespace="project:my-app") as client:
        await client.store(
            "Pydantic v2 requires use_enum_values=True for YAML round-trip",
            tags=["pydantic", "gotcha"],
            importance=0.8,
        )

        # Uses hybrid retrieval when the optional rankers are installed.
        results = await client.recall("pydantic serialization", limit=10)
        high_impact = await client.search(min_importance=0.7, tags=["gotcha"])
        print(results, high_impact)


asyncio.run(main())

MemoryClient also provides store_many() and bulk_store() for batch writes, search_fts() for keyword-only lookup, forget() for deletion, and audit_learning() / review_quarantined() for lifecycle and security workflows.

Conversation memory without an LLM call at write time

# inside `async with MemoryClient(...) as client:`
turns = [
    {"role": "user", "speaker": "Caroline", "content": "I went to a LGBTQ support group yesterday."},
    {"role": "assistant", "speaker": "Melanie", "content": "That's great! What did it look like?"},
]
summary = await client.store_conversation(turns, observed_at="2023-05-08T13:56:00+00:00", session_id="s1")
rows = await client.recall("what did the support group look like", limit=5)

store_conversation() stores every turn verbatim and carries the preceding context_turns (default 1) of the same conversation alongside it, so a reply like "What did it look like?" is retrievable by what it was replying to. No generative LLM is called at ingest time (the optional local embedding model still encodes each turn): the raw turn, its date and its neighbourhood are the evidence, and the reader does the inference at recall time. Feeding a conversation in chunks? Pass the last turns you already stored as preceding=.

Agent Framework Integration

from trw_memory.client import MemoryClient

client = MemoryClient(namespace="project:my-app")

# Register tools with any agent that has register_tool() or tool() API
client.register_tools(agent)

# Or use the auto_recall decorator
@client.auto_recall(query_from="prompt")
async def handle_prompt(prompt: str, recalled_memories: list | None = None) -> str:
    # recalled_memories is automatically injected with relevant context
    recalled_memories = recalled_memories or []
    return f"Found {len(recalled_memories)} relevant memories"

CLI

# Store a learning
trw-memory store "Always use connection pooling for PostgreSQL" --tags db,performance --importance 0.8

# Recall by query
trw-memory recall "database optimization" --limit 5

# Search with filters
trw-memory search --tags security --min-importance 0.7

# Consolidate related entries
trw-memory consolidate --namespace project:my-app --dry-run

# Export/import a namespace's entry data
trw-memory export --format json > memories.json
trw-memory import memories.json --namespace project:new-app

# Forget an entry by ID
trw-memory forget M-abc12345 --namespace project:my-app

# Rebuild the SQLite DB from the cold YAML tier or a snapshot
trw-memory restore --from-cold
trw-memory restore --from-snapshot latest

# Snapshot management (VACUUM INTO rotation)
trw-memory snapshot create --tier daily
trw-memory snapshot list
trw-memory snapshot rotate

# Lint wiki page JSON for missing targets/backlinks/provenance
trw-memory wiki-lint pages.json

# Explicit code index: index, lexical search, and symbol lookup
trw-memory code-index ./src
trw-memory code-search ./src "hybrid_search" --language python --limit 5
trw-memory code-symbol ./src MemoryClient

# Status overview
trw-memory status

Export enumerates the requested namespace (default unless specified), not the whole project or every namespace. Use an unchanged store for a consistent export: pagination is not a snapshot across concurrent writes. JSON/YAML output retains the entry format and is materialized in memory; it is not a streaming database backup and does not include arbitrary project files or stored vector indexes. The separate snapshot commands above serve database snapshot management.

Low-Level Backend Access

from trw_memory.storage.sqlite_backend import SQLiteBackend
from trw_memory.models.memory import MemoryEntry

backend = SQLiteBackend(db_path=".trw/memory.db")
entry = MemoryEntry(id="M-abc12345", content="Use WAL mode for concurrent readers", namespace="default")
backend.store(entry)
results = backend.search("query", top_k=10, namespace="default")

Benchmarks

Every number below is reported with its sample size; confidence intervals and paired tests are given where they were computed. Apart from the mem0 comparison, these are same-harness ablations — retrieval strategies compared on one fixed corpus and query set — not leaderboard claims against other systems. The framework's evidence posture is described in the verification docs; the mem0 comparison's method and scripts live in benchmarks/locomo/.

Single-conversation comparison with mem0 (OSS), using mem0's evaluation suite

Scope first. One LOCOMO conversation of ten, one run per system. The answerer, the judge and mem0's extraction model were all a local 8B llama3.1; that judge was not calibrated against the GPT-class judges behind mem0's published numbers, so compare the two columns with each other, not with mem0's website. mem0 was run as its open-source SDK (mem0ai 2.0.20), not Mem0 Cloud. Results apply to these configurations only.

We ran mem0's open-source evaluation suite (commit 4b61c5d) unmodified against both systems: same dataset parsing, same answer prompt, same LLM judge, same cutoffs, and the same embedding model (all-MiniLM-L6-v2) for both. Conversation 0, n = 152 questions per system, paired by question, trw-memory 0.19 defaults:

Neither paired test detected a statistically significant accuracy difference. That does not establish equivalence or superiority; it means this sample could not tell the two apart.

Ingestion measurements for the same run (419 turns, one machine, single run):

These measurements do not establish total operating cost: recall and answer generation are not included, and "top k" counts stored items, not equal token budgets (a verbatim turn and an extracted fact are different units). Verbatim storage avoids generative rewriting during ingestion; it does not guarantee correct input metadata, retrieval, or answers.

How trw-memory gets there without a generative model at write time: store_conversation() keeps each turn verbatim with its conversational context, and recall does the work (BM25 with stemming + dense vectors + rank fusion + a cross-encoder re-ranker that drops low-confidence rows). Evidence retrieval over all ten LOCOMO conversations (n = 1,540 questions, no LLM in the loop): the gold evidence turn is in the top 10 for 84.0% of questions and in the top 50 for 90.5%.

An additional, non-default configuration (MEMORY_RECALL_RERANK_MIN_SCORE=-5) scored 92.1% at top 10 and 95.4% at top 50 on the same questions. That threshold was chosen after inspecting misses on this same conversation, so it is an exploratory result, not an independently validated improvement. Method, scripts and the paired-comparison tool: benchmarks/locomo/.

Hybrid retrieval beats either ranker alone

On a gold set of real engineering learnings (n = 889 typed queries), Reciprocal Rank Fusion of BM25 + dense vectors outranks either single ranker (the table below is this gold set; point estimates, no intervals computed):

The same direction was observed on a second, independent benchmark (LongMemEval_S, n = 500 questions); those figures are not reproduced here. Fusion earns its keep on the hard questions: exact-match queries are near ceiling for every retriever, so the lift concentrates in the temporal / multi-session discrimination band.

Retrospective retrieval of previously stored duplicates

On TRW's own active learning store (n = 175 near-duplicate "rediscoveries"), the share of duplicates a recall would have caught before re-deriving them — the Preventable Rediscovery Ratio — is far higher for hybrid than for keyword search alone, with non-overlapping 95% CIs:

Hybrid retrieval surfaced more of these previously stored duplicates; this evaluation did not measure whether agents then avoided re-deriving them.

Cross-session recall on constructed tasks

On a controlled recall-dependent benchmark (H1-MEMORY-BENCH), agents with memory solved every task that required recalling a fact established in an earlier session — 58/58 — while agents without memory solved 0/50 (the fact is absent by construction). Paired McNemar p = 3.6×10⁻¹⁵ across 49 matched pairs (exceeds the pre-registered n ≥ 30), replicated on a second model family.

Scope, honestly. This demonstrates the mechanism: cross-session recall lets an agent complete work it otherwise cannot. Whether that compounds into broad, end-to-end coding-task improvement is a separate, still-open question — early SWE-bench single-shot runs (n ≥ 40) produced null. See the verification docs for the full evidence posture.

Throughput (historical single-run baseline, not CI-backed, measured before cross-encoder re-ranking became the default; re-ranking adds roughly 30-300 ms per recall on CPU): sub-millisecond store (p95 ≈ 0.31 ms) and ~116 ms hybrid recall p95 at 1,000 entries; on-disk footprint ≈ 1.2 MB per 1k entries.

Architecture

The engine is organized as a set of focused subpackages under src/trw_memory/. (For the authoritative, always-current layout, browse the source tree directly — file-level listings drift quickly.)

API Reference

Key Modules and Functions

Storage Backends

SQLite (recommended) -- Fast, transactional, supports keyword search, knowledge graph edges, and optional sqlite-vec vector similarity:

from trw_memory.storage.sqlite_backend import SQLiteBackend

backend = SQLiteBackend(db_path=".trw/memory.db")
# Supports: store, get, update, delete, search, count, list_entries,
#           list_namespaces, upsert_vector, search_vectors

YAML -- Human-readable, git-friendly, used as backup during migration:

from trw_memory.storage.yaml_backend import YAMLBackend

backend = YAMLBackend(entries_dir=".trw/learnings")

How hybrid retrieval works: BM25 + vector search + cross-encoder reranking

The hybrid search pipeline combines sparse keyword retrieval with dense semantic search — ensuring strong results for both exact-match queries and conceptually similar queries. Read the full architecture docs.

Query --> BM25 (keyword, rank-bm25) --+
                                       +--> RRF Fusion (k, configurable) --> Ranked Results
Query --> Dense (cosine, sqlite-vec) --+

BM25 drops function words from the query and suffix-stems tokens on both sides ("researched" meets "research"); after fusion a cross-encoder re-ranks the top recall_rerank_candidates (default on, MEMORY_RECALL_RERANK=false to disable for latency-critical paths). The RRF constant k is configurable via MemoryConfig.rrf_k (env MEMORY_RRF_K); the shipped default is tuned by the memory meta-harness loop and may change between releases, so treat the exact value as a default rather than a contract.

The pipeline gracefully degrades: if BM25 is unavailable, only dense search runs (and vice versa). If neither is available, falls back to the storage backend's built-in keyword search (case-insensitive LIKE matching).

Scoring System

Learning utility is computed from multiple signals. Full scoring documentation:

• Q-learning: Exponential moving average updated from outcome events (success/failure/mixed)
• Ebbinghaus forgetting curve: Time-based Ebbinghaus decay applied at query time (not mutated in storage) — entries naturally fade unless reinforced by recall
• Access recency boost: Recently accessed entries score higher
• Impact score: Author-assigned importance (0.0-1.0)
• Bayesian calibration: MACLA calibration for impact score accuracy

Tiered Storage

Hot/warm/cold tiering keeps frequently-used memories fast and archives stale ones. Architecture overview:

The latency column is the design target for the tier lookup itself, not end-to-end recall latency (hybrid recall with re-ranking is slower; see Benchmarks). Store/recall operations keep Hot/Warm in sync, Cold-tier hits are promoted back to Warm within the same recall, and TierManager.sweep() applies the configurable archive/purge policy when callers trigger a lifecycle sweep.

Security

MCP memory server

The MCP server ships with the core install (fastmcp is a core dependency):

trw-memory-server  # Starts MCP server (stdio transport)

To wire it into an MCP client (Claude Code, Cursor, Claude Desktop and others use this shape):

{
  "mcpServers": {
    "memory": { "command": "trw-memory-server" }
  }
}

Loopback daemon (serve http)

trw-memory-server serve http runs one process per operating-system user, serving the same MCP tool surface over streamable-http on 127.0.0.1 with a per-user bearer token. The port is ephemeral by default and published in a 0600 daemon.json beside the store, so clients discover it rather than hardcode it.

Trust boundary: one principal. The daemon authenticates the token file, not the caller. Anyone who can read ~/.trw/memory/daemon-token is fully authorized for every namespace in that store; a namespace argument selects scope, not permission. The boundary is therefore the user account, and that is deliberate — this transport is for one user's agents and applications, not for mutually distrusting tenants.

Concurrency: four workers. Each served memory_recall, memory_store and memory_maintain call runs its synchronous work in a bounded thread pool (OFFLOAD_MAX_WORKERS = 4 in daemon/_offload.py), opening and closing its own SQLite connection inside the worker. Four calls make progress at once; the fifth queues, and that queue is unbounded. A request that is cancelled after it starts still runs to completion — the result is discarded, not the work.

Shutdown. SIGTERM and SIGINT drain the worker pool, remove the discovery record, and then let the signal take its default disposition, so a service manager stopping the daemon does not leave clients pointed at a dead endpoint. The record is only ever removed when it names this process and the start time this process wrote, so a slow exit cannot delete a successor's record.

Maintenance. A daemon has no session end, so decay, consolidation and WAL checkpointing never run on their own. memory_maintain(namespace) triggers them and records last_attempted_at / last_maintained_at per namespace in maintenance.json beside the store. Scope is not uniform: consolidation is namespace-scoped, while the decay pass and the WAL checkpoint act on the whole store.

Recall is bounded. Each namespace contributes at most max(limit * 5, hybrid_search_candidate_pool_size) entries (default 1000) to a search, chosen as the most recently updated rows. On a larger namespace, older entries are not searched, and an empty result is not evidence of absence. Raising MEMORY_HYBRID_SEARCH_CANDIDATE_POOL_SIZE widens it at a real cost: measured on a 6500-row namespace, warm recall was 139.6 ms at 1000 and 1045.8 ms at 10000.

Integration with trw-mcp

trw-mcp is the MCP server layer of TRW Framework — it exposes a suite of tools, skills, and agents to Claude Code and other AI coding tools (see the trw-mcp README for current counts). trw-memory serves as its memory backend:

• trw_learn delegates to SQLiteBackend.store() via memory_adapter.py (YAML dual-write as backup)
• trw_recall delegates to SQLiteBackend.search() / list_entries() as the sole query path
• Scoring functions (compute_utility_score, update_q_value, apply_time_decay, bayesian_calibrate) are canonical in trw-memory and re-exported by trw-mcp
• One-time YAML-to-SQLite migration runs automatically on first access
• Optional vector search via LocalEmbeddingProvider + rrf_fuse when sentence-transformers is installed

Read more about the full TRW Framework architecture.

Telemetry and network behavior

trw-memory is local-first: with the default configuration all data lives in a local SQLite store (and an optional YAML sidecar). It makes no outbound network calls except the optional model downloads below (embedding model and cross-encoder re-ranker). There is no usage tracking or content phone-home.

What can touch the network, when, and how to turn it off

A warm cache performs no Hub request, and embedding egress is independent of the consent flags. A fetch is attempted only when the cached snapshot is incomplete or absent and no offline switch is engaged; in exactly that case one structured disclosure log names the host and the switch that would block it. learning_sharing_enabled and platform_telemetry_enabled govern learning-content publishing and usage telemetry respectively — neither gates the embedding model fetch. Embedding egress is governed by the local cache, the offline switches, and local_only.

sync_enabled defaults false, so the engine performs no remote sync out of the box even though local_only itself defaults false. Setting local_only: true is the hard-block: an @model_validator forces sync_enabled=False, clears sync_namespace/platform_url, and pins rbac_mode="local", so no remote-capable surface can be re-enabled while it is set.

With an offline switch engaged (TRW_OFFLINE / HF_HUB_OFFLINE) or local_only: true, the standalone engine loads the embedding model with local_files_only=True; if the model is not already cached it raises a clear LocalOnlyViolationError telling you how to pre-download (this is the behaviour in both cases — local.py does not silently fall back to keyword-only recall here). The graceful "degrade to keyword-only, no crash" path is provided one layer up by trw-mcp's embedder wrapper, which catches that error; a direct trw-memory caller that wants keyword-only recall under an offline switch should pre-download the model or run without the [embeddings] extra installed.

Environment-variable inventory

Security defaults

Enterprise hardening recipe

export TRW_OFFLINE=1   # block the huggingface.co model download (local_files_only)

# MemoryConfig
local_only: true       # hard-block all remote sync + model download

For hybrid recall offline, populate the model cache before enabling either switch, in the same environment: python -c "from sentence_transformers import SentenceTransformer, CrossEncoder; SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2'); CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')". Otherwise the first embedding load raises LocalOnlyViolationError (an uncached re-ranker is skipped silently and recall keeps fusion order). To run keyword-only without that error, omit the [embeddings] extra entirely. Verify the on-disk memory.db is mode 0600 and that no outbound connection is attempted on first use.

Migration notes

Retired hypothetical expansion (unreleased)

HyPE question generation and HyDE query expansion are removed. Ordinary embeddings, lexical/hybrid recall, code/wiki references and Distill data are not removed. Delete imports of QuestionGenerator and NoOpQuestionGenerator. Remove question_generator, query_expansion, and collapse_hype arguments, and the hype_enabled, hype_questions_per_entry, hype_min_question_chars settings (including memory_ aliases and environment/YAML entries).

Explicit neutral legacy settings (False, 3, 8) and API arguments (None for the generator, None/blank expansion, False for collapse) warn temporarily; activation, nondefault values and invalid types fail before the operation. Retired settings no longer appear in emitted configuration. These tombstones will disappear in the next declared breaking API release after the retirement release; that release's notes must announce their removal.

Existing derived question vectors are not knowledge records. Recall ignores them by requiring canonical membership, without excluding real IDs that happen to end in #hype0. Normal update/forget removes only namespace-owned noncanonical siblings of the selected canonical parent. Orphan/unknown vector rows remain untouched for a future canonical-only index rebuild. No startup purge occurs.

Optional legacy-vector maintenance on a disposable snapshot

Stop old-version writers first; they can regenerate retired vectors. Preserve a verified backup using SQLite's online backup API (the approach in storage/_schema_backup.py), not a copy of a live database without its WAL. Do not overwrite canonical writes made since a snapshot to recover optional vectors. The retirement itself changes no schema or historical migration.

This recipe is for an existing disposable unencrypted snapshot, not a live store. Choose its original embedding dimension, namespace and parent IDs explicitly; opening SQLiteBackend can perform normal schema initialization. Encrypted stores require their existing key-aware backup/open procedure instead. apply = False only enumerates selected vectors; changing it to True removes those derived vectors atomically. It never deletes canonical records or other namespaces. No vectors installed means unavailable, not a successful cleanup.

from pathlib import Path
from trw_memory.storage.sqlite_backend import SQLiteBackend

snapshot = Path("/absolute/path/to/disposable-snapshot.db")
if not snapshot.is_file():
    raise FileNotFoundError(snapshot)
namespace = "default"  # explicitly selected, locally authorized namespace
parents = ["selected-parent-id"]
apply = False
backend = SQLiteBackend(snapshot, dim=384)  # use this snapshot's dimension
try:
    if not backend.supports_vectors():
        raise RuntimeError("legacy cleanup unavailable: sqlite-vec required")
    with backend.transaction():
        for parent_id in parents:
            siblings = backend.hype_sibling_ids(parent_id, namespace=namespace)
            print(parent_id, siblings)
            if apply:
                backend.delete_hype_siblings(parent_id, namespace=namespace)
finally:
    backend.close()

Cleanup is idempotent; interruption rolls the transaction back. Package rollback can reopen the same canonical store; restoring previous optional ranking also requires its matching derived-index snapshot. Never discard newer canonical data for that purpose. Internal cleanup helpers will be removed once the supported store floor rejects pre-retirement stores unless canonical-only vector rebuilding has been verified; ordinary orphan-index handling then owns residual derived data.

Platform and interpreter notes

Supported interpreters

trw-memory is tested on CPython 3.10 through 3.14 (this repository's own development interpreter is CPython 3.14.7). One property of the interpreter matters beyond the version: its bundled SQLite. WAL space is only RECLAIMED on SQLite >= 3.51.3 (or the 3.44.6 / 3.50.7 backports) — below that, storage/_wal_checkpoint.py coerces resetting checkpoints to PASSIVE, which is correct and safe but lets the -wal file grow without shrinking. Check yours with python -c "import sqlite3; print(sqlite3.sqlite_version)"; on macOS, Homebrew's current Python ships a qualifying build, and trw-mcp doctor names the qualifying interpreters it finds.

The engine is SELECTED at import by storage/_dbapi.py, which ranks the interpreter's SQLite against an installed pysqlite3 on (carries the fix, version) and never replaces a newer engine with an older wheel. The optional [sqlite-fix] extra pulls pysqlite3-binary on x86_64 Linux only — no published wheel currently bundles a qualifying SQLite, so it is an engine override, not a fix.

Platform notes

• SQLite driver — pysqlite3-binary is no longer a runtime dependency on any platform; it moved to the optional [sqlite-fix] extra, marked for x86_64 Linux (the only platform it publishes a wheel for). It used to be a hard Linux dependency, which made aarch64 Linux installs fail outright while delivering SQLite 3.51.1 — below the 3.51.3 fix it existed to provide. The runtime probe in storage/_dbapi.py, not the dependency name or the package version, decides and reports which engine is active.
• Vector search is optional — [vectors] (sqlite-vec) and [embeddings] (sentence-transformers) are optional extras. When they are unavailable the retrieval pipeline degrades gracefully to BM25 and/or the backend's built-in keyword search rather than failing.

Development

# Install dev dependencies
pip install -e ".[dev]"

# Run full test suite (>=85% coverage required — see fail_under in pyproject.toml)
python -m pytest tests/ -v --cov=trw_memory --cov-report=term-missing

# Type checking (mypy --strict across the package)
python -m mypy --strict src/trw_memory/

# Targeted testing
python -m pytest tests/test_client_*.py -v
python -m pytest tests/test_retrieval_*.py -v
python -m pytest tests/test_storage_sqlite_*.py -v

Quality bar: a broad pytest suite, mypy --strict clean, and a coverage floor of 85% (fail_under in pyproject.toml).

Optional Dependencies

There is no [llm] extra and no LLM-backed consolidation. Consolidation summarises a cluster with a longest-content heuristic; an earlier revision of this table advertised [llm]/anthropic "LLM-augmented consolidation", which this package never implemented. The [langchain], [llamaindex], [crewai] and [all-integrations] extras and their adapter modules were removed as unused surface — see CHANGELOG.md [Unreleased] Removed.

Entry Points

FAQ

What is trw-memory?

A persistent, local-first memory engine for AI agents. It stores memories in SQLite and recalls them with keyword search, or, with the retrieval extras installed, hybrid retrieval (BM25 + dense vectors, fused with Reciprocal Rank Fusion, then a cross-encoder re-ranker). It ships a Python SDK (MemoryClient), a CLI (trw-memory), and an MCP server (trw-memory-server). You do not need TRW Framework to use it.

Does trw-memory need an LLM to store memories?

No. store_conversation() stores every turn verbatim and calls no generative LLM at ingest time; the reader does the inference at recall time. There is also no [llm] extra and no LLM-backed consolidation: consolidation summarises a cluster with a longest-content heuristic. Dense retrieval uses a local sentence-transformers embedding model (all-MiniLM-L6-v2) from the optional [embeddings] extra.

Does it work offline?

Yes. With the default configuration all data is local and remote sync is off. TRW_OFFLINE=1 / HF_HUB_OFFLINE=1 or local_only: true force local_files_only=True for the embedding and re-ranker models. If the embedding model is not already cached when one of those switches is set, the first embedding load raises LocalOnlyViolationError: pre-download the model, or omit the [embeddings] extra to run keyword-only. See Telemetry and network behavior.

How does trw-memory compare to mem0?

One small, scoped comparison exists. Using mem0's open-source evaluation suite, unmodified, on LOCOMO conversation 0 (n = 152 questions per system, paired by question, one run each): trw-memory 91.4% [85.9, 94.9] vs mem0 (OSS) 88.2% [82.1, 92.4] at top 10 (McNemar p = 0.38), and 91.4% [85.9, 94.9] vs 92.1% [86.7, 95.4] at top 50 (p = 1.00). Neither test detected a statistically significant difference, which does not establish equivalence or superiority. In that run mem0 made ~2 generative LLM calls per turn to ingest the 419-turn conversation (1 h 28 min); trw-memory made none (~75 s); those figures are ingestion only, not total operating cost. Conditions: a local 8B llama3.1 as answerer, judge and mem0's extraction model, and mem0 run as its open-source SDK, not Mem0 Cloud. See the benchmark section.

Does recall search every stored memory?

Not on a very large namespace. Each namespace contributes at most max(limit * 5, hybrid_search_candidate_pool_size) entries (default 1000) to a search, chosen as the most recently updated rows, so on a larger namespace older entries are not searched and an empty result is not evidence of absence. MEMORY_HYBRID_SEARCH_CANDIDATE_POOL_SIZE widens it at a latency cost. With re-ranking on (the default), recall(limit=N) can also return fewer than N rows: results the cross-encoder scores below MEMORY_RECALL_RERANK_MIN_SCORE are dropped.

Where is my data stored?

In .memory/ relative to the current directory by default (override with MEMORY_STORAGE_PATH), as a local SQLite database per namespace plus an optional YAML sidecar. Nothing leaves the machine unless you enable remote sync.

Can I use it as an MCP memory server?

Yes. Run trw-memory-server (stdio transport), or trw-memory-server serve http for a per-user loopback daemon. See MCP memory server.

What happens if sqlite-vec or sentence-transformers is not installed?

[vectors] and [embeddings] are optional extras. When they are unavailable the retrieval pipeline degrades gracefully to BM25 and/or the backend's built-in keyword search rather than failing.

Does agent memory improve coding-task outcomes?

That is an open empirical question. On a controlled recall-dependent benchmark (H1-MEMORY-BENCH), agents with memory solved 58/58 tasks that required a fact from an earlier session and agents without memory solved 0/50 (the fact is absent by construction), which demonstrates the mechanism. Early SWE-bench single-shot runs (n ≥ 40) produced null. See Knowledge compounding, measured.

What license is trw-memory under?

Business Source License 1.1: source-available, free for non-competing use, converting to Apache 2.0 on 2030-03-21. The package is alpha.

License

Business Source License 1.1 -- source-available, free for non-competing use. Converts to Apache 2.0 on 2030-03-21.

Built by Tyler Wall · TRW Framework · Documentation · License