context-fabrica 0.4.0

Governed agent memory with hybrid retrieval, temporal recall, and graph reasoning

context-fabrica

A governed memory layer built specifically for coding knowledge—giving agents durable recall, relation awareness, and evidence they can explain.

Hybrid retrieval, graph reasoning, temporal recall, provenance-backed synthesis, and policy controls in one composable library.

Getting Started | Architecture | Examples | Contributing | Releasing

---

The Problem

Flat vector memory fails autonomous coding agents. Without the ability to trace provenance, relate concepts across a codebase, or understand temporal validity, agents cannot distinguish between deeply vetted architectural rules and outdated drafts. Session recall isn't enough—agents need memory they can govern, trust, and safely update over time.

What context-fabrica Does

context-fabrica is a composable library that combines semantic retrieval, graph traversal, and temporal recall so agents can genuinely reason about their memory. By treating provenance, validity, curation stage, and supersession as first-class data rather than afterthought metadata, it enables agents to justify their answers rather than just retrieving vaguely similar text.

Query: "How does PaymentsService interact with LedgerAdapter?"

  Semantic score ──── 0.72  (embedding similarity + BM25 lexical boost)
  Graph score ─────── 0.85  (2-hop traversal: PaymentsService → depends_on → LedgerAdapter)
  Temporal score ──── 0.00  (not a time-scoped query)
  Recency score ───── 0.91  (ingested 3 hours ago)
  Confidence score ── 0.80  (from design-doc source)
                      ────
  Final score ─────── 0.66  (hybrid weighted fusion)
  Rationale: [semantic_match, graph_relation, recent, high_confidence]

Every query returns scored results with full breakdowns — your agents can reason about why a memory was relevant, not just that it was.

---

Perfect for Self-Learning & Self-Improving Agents

context-fabrica was designed to give coding agents the scaffolding they need to reason about their own growth and incrementally improve over time:

• Observation Synthesis: Agents can piece together raw, disparate facts over time and explicitly synthesize them into foundational insights (e.g., "The auth framework in this repo is unstable around token refresh").
• Memory Tiers & Promotion: "I saw something once" is not "This is a factual rule." Agents can log low-confidence observations in a staged tier. If the observation proves true repeatedly, it gets promoted to canonical or extracted as a reusable pattern applied to future tasks.
• Supersession & Error Correction: When an agent realizes a past assumption was incorrect, it uses supersession chains (soft invalidation). This means it remembers both what it previously thought and why it updated its understanding, rather than destructively overwriting past knowledge.
• Temporal Recall: Agents can explicitly time-scope memories to avoid confusing legacy system knowledge with the current state of a codebase.

---

Core vs Extensible

context-fabrica separates what is core (the retrieval model, memory semantics, and governance) from what is pluggable (storage backends, embedders, and entity extraction).

  ┌──────────────────────────────────────────────────────────┐
  │                      CORE (fixed)                        │
  │                                                          │
  │  DomainMemoryEngine        Hybrid scoring formula        │
  │  KnowledgeRecord model     Memory tiers & promotion      │
  │  Validity windows          Provenance tracking           │
  │  Temporal recall           Namespace policies            │
  │  BM25 lexical index        Knowledge graph traversal     │
  └──────────────────────────────────────────────────────────┘
                          │
              ┌───────────┼───────────┐
              ▼           ▼           ▼
  ┌──────────────┐ ┌───────────┐ ┌──────────────┐
  │ RecordStore  │ │ Embedder  │ │ GraphStore   │
  │  (protocol)  │ │ (protocol)│ │  (protocol)  │
  └──────┬───────┘ └─────┬─────┘ └──────┬───────┘
         │               │              │
   ┌─────┴─────┐   ┌─────┴─────┐  ┌────┴─────┐
   │  SQLite   │   │   Hash    │  │   Kuzu   │
   │  Postgres │   │ FastEmbed │  │  Neo4j*  │
   │  Custom   │   │ Sentence  │  │  Custom  │
   └───────────┘   │ Transformr│  └──────────┘
                   │  Custom   │    * planned
                   └───────────┘

Core — the retrieval model, ranking formula, memory lifecycle, and governance primitives. These define what context-fabrica is and are not meant to be swapped out.

Extensible — storage backends, embedding providers, and graph stores are pluggable via Python protocols. Implement the interface, pass it in.

---

Storage Options

Pick the backend that matches your scale. No code changes needed — the HybridMemoryStore API is the same regardless of backend.

Backend	Dependencies	Server required?	Best for
SQLite (built-in)	None (stdlib)	No	Local dev, single-agent, getting started
Postgres + pgvector	psycopg, pgvector	Yes	Production, multi-agent, teams
Kuzu (optional add-on)	kuzu	No	Graph-heavy traversal at scale
Custom	You decide	You decide	Bring your own (LanceDB, DuckDB, etc.)

SQLite — zero setup, no server

pip install context-fabrica

from context_fabrica import HybridMemoryStore, SQLiteRecordStore

store = HybridMemoryStore(store=SQLiteRecordStore("./memory.db"))
store.bootstrap()

# Same API as Postgres — write, query, promote, search
store.write_text(record)
results = store.semantic_search(query_embedding, top_k=5)

SQLite stores records, chunks, embeddings, relations, and promotions in a single file. Semantic search uses brute-force cosine similarity — fast enough for local dev and single-agent workloads up to ~50k records.

Postgres + pgvector — production scale

pip install "context-fabrica[postgres,kuzu,fastembed]"

If you are working from a local clone instead of PyPI:

python -m pip install .
python -m pip install -r requirements-v2.txt

from context_fabrica import HybridMemoryStore, HybridStoreSettings, PostgresSettings, KuzuSettings

store = HybridMemoryStore(
    HybridStoreSettings(
        postgres=PostgresSettings(dsn="postgresql:///context_fabrica"),
        kuzu=KuzuSettings(path="./var/graph"),
    )
)
store.bootstrap()

Postgres handles records, chunks, HNSW-indexed vector search, validity windows, and provenance. Kuzu is optional — if you don't need multi-hop graph traversal at scale, skip it.

Postgres without Kuzu

from context_fabrica import HybridMemoryStore
from context_fabrica.storage.postgres import PostgresPgvectorAdapter

store = HybridMemoryStore(
    store=PostgresPgvectorAdapter(PostgresSettings(dsn="postgresql:///context_fabrica"))
)
store.bootstrap()
# No graph projection — relations still stored in Postgres, just no Kuzu traversal

Bring your own backend

Implement the RecordStore protocol and pass it in:

from context_fabrica.adapters import RecordStore

class MyLanceDBStore:
    """Implements RecordStore protocol."""
    def bootstrap(self) -> None: ...
    def upsert_record(self, record: KnowledgeRecord) -> None: ...
    def fetch_record(self, record_id: str) -> KnowledgeRecord | None: ...
    def replace_chunks(self, record_id: str, chunks: list) -> None: ...
    def replace_relations(self, record_id: str, relations: list) -> None: ...
    def record_promotion(self, source_id: str, target_id: str, reason: str, promoted_at: datetime) -> None: ...
    def semantic_search(self, query_embedding: list[float], *, domain: str | None, top_k: int) -> list[QueryResult]: ...
    def enqueue_projection(self, record_id: str) -> None: ...

store = HybridMemoryStore(store=MyLanceDBStore(), graph=MyGraphStore())  # graph is optional

---

Key Features

Feature	Description
Hybrid retrieval	Embedding cosine similarity + BM25 lexical boost + graph traversal, fused into one score
Temporal retrieval	Time-aware recall for queries like "what happened in June 2025?"
Knowledge graph	Entity-relation extraction with multi-hop traversal (configurable depth)
Curated memory tiers	staged (draft) -> canonical (reviewed) -> pattern (reusable)
Observation synthesis	Explicitly synthesize provenance-backed observation records from multiple facts
Soft invalidation	Validity windows (valid_from/valid_to) instead of hard deletes
Promotion provenance	Track when, why, and by whom records were promoted
Namespace policies	Per-namespace retrieval controls for hops, confidence floor, source allowlists, and reranking
Caller-provided extraction	Pass your own entities and relations from an upstream LLM — or use built-in heuristics
Optional reranking	Add a second-stage reranker on top of hybrid or RRF retrieval when precision matters
Scoring modes	hybrid (default), embedding-only, bm25-only, or rrf
Pluggable storage	SQLite (built-in), Postgres + pgvector, or bring your own via RecordStore protocol
Pluggable embedders	HashEmbedder (zero-dep), FastEmbed, SentenceTransformers, or bring your own via Embedder protocol
Optional graph store	Kuzu ships as default, but graph projection is fully optional
Framework-agnostic	Not locked to LangChain, CrewAI, or any orchestrator

Quick Start

Install from PyPI:

python -m pip install "context-fabrica[postgres,kuzu,fastembed]"

Then bootstrap and verify:

context-fabrica-bootstrap --dsn "postgresql:///context_fabrica"
context-fabrica-doctor --dsn "postgresql:///context_fabrica"
context-fabrica-demo --dsn "postgresql:///context_fabrica" --project

from context_fabrica import DomainMemoryEngine, NamespacePolicy, ScoringWeights, TokenOverlapReranker
from context_fabrica.models import Relation

engine = DomainMemoryEngine(
    reranker=TokenOverlapReranker(),
    namespace_policies={
        "payments": NamespacePolicy(
            min_confidence=0.75,
            source_allowlist=("design-doc", "runbook"),
            rerank_top_n=5,
        )
    },
)  # or DomainMemoryEngine(scoring="embedding")

# Ingest with automatic entity/relation extraction
engine.ingest(
    "PaymentsService depends on LedgerAdapter and calls RiskGateway.",
    source="design-doc",
    domain="fintech",
    confidence=0.8,
)

# Or provide your own entities/relations (e.g. from an upstream LLM)
engine.ingest(
    "The auth service validates tokens before routing to the API gateway.",
    source="architecture-review",
    domain="platform",
    confidence=0.9,
    entities=["auth_service", "api_gateway", "token_validator"],
    relations=[
        Relation("auth_service", "calls", "api_gateway"),
        Relation("auth_service", "uses", "token_validator"),
    ],
)

# Query with full score breakdown
results = engine.query("How does PaymentsService interact with LedgerAdapter?", top_k=3)
for hit in results:
    print(f"{hit.record.record_id}  score={hit.score:.2f}  {hit.rationale}")

# Temporal recall
incident = engine.ingest(
    "Quarterly incident review happened in June 2025.",
    source="incident",
    domain="platform",
    confidence=0.9,
    record_id="incident-june",
)
time_scoped = engine.query("What happened in June 2025?", top_k=3)

# Observation synthesis — combine multiple facts into one provenance-backed record
engine.ingest("AuthService depends on TokenSigner.", record_id="f1", confidence=0.8)
engine.ingest("TokenSigner rotates keys daily.", record_id="f2", confidence=0.9)
observation = engine.synthesize_observation(["f1", "f2"], record_id="obs-1")
assert observation.metadata["derived_from"] == ["f1", "f2"]

Architecture

                    +------------------+
                    |   Agent / CLI    |
                    +--------+---------+
                             |
                    +--------v---------+
                    | DomainMemoryEngine|
                    |  (in-process)     |
                    +--------+---------+
                             |
         +----------+--------+--------+----------+
         |          |                 |           |
  +------v---+ +---v--------+ +-----v------+ +--v--------+
  | Embedding| | BM25       | | Knowledge  | | Temporal  |
  | Similarity | Lexical    | | Graph      | | Overlap   |
  +------+---+ | Index      | +-----+------+ +--+--------+
         |     +------------+       |            |
    (pluggable)              multi-hop BFS  occurrence
                              with decay    windows

Scoring formula (default weights, normalized to sum to 1.0): 0.42 * semantic + 0.25 * graph + 0.15 * temporal + 0.10 * recency + 0.07 * confidence

Where semantic = 0.70 * embedding + 0.30 * BM25 in hybrid mode. Weights are always normalized at query time, so custom values don't need to sum to 1.0.

Temporal scoring is neutral unless the query or record carries time information.

Namespace Policies

Use namespace policies when one team or agent needs stricter retrieval than another without forking the engine:

from context_fabrica import DomainMemoryEngine, NamespacePolicy, ScoringWeights

engine = DomainMemoryEngine(
    namespace_policies={
        "production-ops": NamespacePolicy(
            weights=ScoringWeights(semantic=0.45, graph=0.25, temporal=0.25, recency=0.10, confidence=0.10),
            min_confidence=0.8,
            source_allowlist=("runbook", "incident", "design-doc"),
            default_hops=1,
            rerank_top_n=8,
        )
    }
)

Persistent Storage

  Agent
    |
    v
  HybridMemoryStore ─────── same API regardless of backend
    |
    ├── RecordStore (protocol)
    │     ├── SQLiteRecordStore     ← zero setup, single file
    │     ├── PostgresPgvectorAdapter ← production, HNSW indexing
    │     └── YourCustomAdapter     ← implement the protocol
    │
    └── GraphStore (protocol, optional)
          ├── KuzuGraphProjectionAdapter ← embedded graph
          └── YourCustomGraph            ← implement the protocol

When using Postgres, the projection worker uses LISTEN/NOTIFY for low-latency graph projection job pickup with polling fallback.

Memory Tiers

Not every agent output deserves canonical memory. context-fabrica models three tiers:

raw observation ──> staged ──> reviewed ──> canonical
repeated pattern ──> mined ──> pattern

Tier	Purpose	In default retrieval?
staged	Draft notes, low-confidence observations	No
canonical	Reviewed facts, trusted knowledge	Yes
pattern	Reusable templates and extracted patterns	Yes

# Low-confidence notes are auto-staged
draft = engine.ingest("TODO: investigate flaky auth refresh", confidence=0.4)
assert draft.stage == "staged"  # excluded from queries

# Promote after review
engine.promote_record(draft.record_id)  # now canonical, queryable

Embedder Options

Embedder	Dimensions	Dependencies	Quality
HashEmbedder (default)	1536	None	Deterministic hashing, good for dev/testing
FastEmbedEmbedder	384	fastembed	Lightweight ML, good balance
SentenceTransformerEmbedder	384+	sentence-transformers	Production-quality semantic similarity

from context_fabrica import DomainMemoryEngine, SentenceTransformerEmbedder

# Production setup with real embeddings
engine = DomainMemoryEngine(
    embedder=SentenceTransformerEmbedder(),
    scoring="hybrid",
)

CLI

# Query from JSONL dataset
context-fabrica --dataset records.jsonl --query "How is TokenSigner connected?" --top-k 5

# Postgres operations
context-fabrica-bootstrap --dsn "postgresql:///context_fabrica"
context-fabrica-doctor --dsn "postgresql:///context_fabrica"
context-fabrica-demo --dsn "postgresql:///context_fabrica" --project

# Projection worker
context-fabrica-projector --once            # process pending jobs
context-fabrica-projector --status          # queue summary
context-fabrica-projector --retry-failed    # requeue failed jobs

# Project memory bootstrap
context-fabrica-project-memory bootstrap --root .

Where It Fits

Good fit:

• Coding agents that need durable codebase/domain memory
• Multi-agent systems that share a canonical knowledge layer
• Orchestration systems wanting inspectable, auditable memory
• Control-plane UIs that need evidence, freshness, and relation visibility

Not a fit:

• Pure chatbot session memory
• Replacement for your agent runtime/orchestrator
• Generic BI or human-only knowledge portal

Governance Primitives

Primitive	Purpose
valid_from / valid_to	Temporal validity windows, enables as-of queries
occurred_from / occurred_to	Event-time windows for time-scoped recall
invalidate_record()	Soft deletion with reason tracking
stage / kind	Promotion routing and curated retrieval
reviewed_at	Promotion auditability
confidence	Trust prior in ranking
source / metadata	Provenance for policy gates
namespace / NamespacePolicy	Tenant isolation plus namespace-specific retrieval controls
supersedes	Record replacement chains

Project Structure

src/context_fabrica/
  engine.py          # In-process hybrid retrieval engine (core)
  models.py          # KnowledgeRecord, Relation, QueryResult (core)
  adapters.py        # RecordStore, GraphStore, Embedder protocols (core)
  policy.py          # Memory tier routing and promotion (core)
  temporal.py        # Time-range extraction and temporal overlap scoring
  synthesis.py       # Provenance-backed observation synthesis
  reranking.py       # Optional second-stage rerankers
  entity.py          # Entity/relation extraction heuristics (core, bypassable)
  index.py           # BM25 lexical index (core)
  graph.py           # In-memory knowledge graph with BFS traversal (core)
  embedding.py       # Embedder adapters: Hash, FastEmbed, SentenceTransformer (pluggable)
  storage/
    sqlite.py        # SQLite record store — zero deps (pluggable)
    postgres.py      # Postgres + pgvector adapter with LISTEN/NOTIFY (pluggable)
    kuzu.py          # Kuzu graph projection adapter (pluggable, optional)
    hybrid.py        # HybridMemoryStore — orchestrates any RecordStore + GraphStore
    projector.py     # Background projection worker
tests/               # pytest suite, including live Postgres coverage
docs/                # Architecture docs and getting-started guide
examples/            # Runnable usage examples
sql/                 # Postgres bootstrap and smoke test SQL

Development

git clone https://github.com/TaskForest/context-fabrica.git
cd context-fabrica
python -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
pytest

References

• GraphRAG — pipeline architecture for graph-enhanced retrieval
• Graphiti — hybrid retrieval with temporal edges
• Neo4j GraphRAG — hybrid graph retriever
• Mem0 — soft-invalidation patterns
• Elasticsearch RRF — reciprocal rank fusion

License

MIT