memotether 0.0.4

Local-first memory layer for LLMs — atomic-memory extraction, hybrid search, and an MCP server for any tool-using agent.

Memotether

A local-first memory layer for LLMs. Memotether ingests text, extracts atomic memories alongside the raw source, and exposes search across keyword / fuzzy / vector / hybrid modes — packaged behind an OpenAI-compatible tool surface.

Install

The fastest path for opencode users is three commands. The MCP server ships with sensible defaults (OpenRouter base URL, qwen/qwen3.6-flash extraction model, local nomic embedder, 16k-char chunks), so the only thing you have to supply is your OpenRouter key:

pipx install "memotether[local-embed]"
export OPENROUTER_API_KEY=sk-or-...   # add to ~/.zshrc / ~/.bashrc so opencode inherits it
memotether install opencode           # writes ~/.config/opencode/opencode.json
# restart opencode → /mcp lists "memotether"

For project-scoped install (writes ./opencode.json and drops AGENTS.md so opencode picks up the agent prompt):

memotether install opencode --project

Other install paths:

# No install — uvx fetches each launch
uvx --from "memotether[local-embed]" memotether-mcp

# Library use / development
pip install -e ".[local-embed]"

Override any default via env (or via .env next to where you launch the server):

OPENROUTER_API_KEY=...                                     # required
MEMOTETHER_MODEL=qwen/qwen3.6-flash                        # default
MEMOTETHER_BASE_URL=https://openrouter.ai/api/v1           # default
MEMOTETHER_MAX_CHUNK_CHARS=16000                           # default
MEMOTETHER_EMBEDDER=nomic                                  # default; "none" to disable vector search
MEMOTETHER_RECONCILE_MODEL=anthropic/claude-haiku-4.5      # optional; falls back to MEMOTETHER_MODEL
MEMOTETHER_FALLBACK_MODEL=xiaomi/mimo-v2.5-pro             # optional

How it works

Ingestion

source text  →  chunk(~16k chars)  ─┬─→  LLM extract  →  atomic memories  →  embed  →  memories tier
                                    │                          │
                                    │                          └─→  reconcile pass (LLM)
                                    │                                flags contradictions,
                                    │                                links via supersedes
                                    │
                                    └─→  raw text  →  embed  →  chunks tier

One read, two indexes. The text gets chunked once and each chunk follows two paths in parallel:

1. Memory tier (lossy). An LLM call per chunk extracts ~10–20 atomic propositions, each typed as one of:

   Universal:

   • fact — stable proposition about an entity
   • preference — stated like, dislike, or policy
   • event — something that happened or will happen at a time
   • relationship — entity-to-entity link
   • procedure — how to do something
   • open_question — unresolved item awaiting resolution
   • quote — a distinctive verbatim line worth preserving

   Engineering-flavored:

   • code_decision — design / implementation choice and its rationale
   • bug — defect symptom, location, and cause if known
   • api_contract — function signature, type, schema, CLI flag, env var
   • requirement — hard constraint the code must satisfy
   • todo — agreed follow-up work for later

   Each memory is one self-contained statement (e.g. "Bertram flees to Florence to serve the Duke.") plus its entities, source turn ids, and confidence. Each memory is embedded and stored. The extractor picks per memory based on what the content is about, so a single store can hold both conversational and coding memories — there's no mode toggle.

2. Chunk tier (verbatim). The raw chunk text is stored unchanged, embedded, and keyword-indexed. This is the safety net for queries that target detail the memory layer compressed away.

3. Reconciliation. After all chunks are extracted, memories are clustered by shared entity (union-find on entity overlap). Each cluster is sent to a second LLM call that flags contradictions — "X moved to Boston" later gets superseded by "X moved to Seattle." Older memories aren't deleted; they're linked via a supersedes table, so retrieval filters them out by default but can opt back in.

Both tiers live in the same SQLite database. Each has three indexes:

• FTS5 with porter stemming for keyword search
• FTS5 with trigrams for fuzzy / typo-tolerant search
• BLOB column with float32 embedding vectors for cosine similarity

Retrieval

query  →  pick tier(s)  →  pick mode(s)  →  top-k candidates  →  LLM synthesize  →  answer
            │                  │
            ├─memories         ├─keyword (FTS5 porter, BM25 ranked)
            ├─chunks           ├─fuzzy   (FTS5 trigram, BM25 ranked)
            └─both             ├─vector  (cosine over float32 embeddings)
                               └─hybrid  (reciprocal rank fusion over the three)

Three knobs:

• tier picks which side of the store to search. memories is fast and cheap, returns compressed propositions. chunks returns raw text. both queries both and concatenates results.
• mode picks the search algorithm. keyword finds stemmed lexical matches. fuzzy handles typos via character trigrams. vector handles semantic similarity. hybrid runs all three and merges via RRF — no score normalization needed.
• k is the result count. Top-k flows into a final synthesis step.

Synthesis is constrained. The synthesizer is told to use ONLY the retrieved context. If the context doesn't contain the answer, it must respond with exactly "I don't know" — not guess, not hallucinate.

Usage

As a tool surface (OpenAI-compatible)

from memotether import Memotether
import json

with Memotether.from_env("memories.db", embedder="nomic") as mt:
    resp = client.chat.completions.create(
        model="...", messages=messages, tools=mt.tools,
    )
    for call in resp.choices[0].message.tool_calls or []:
        result = mt.dispatch(call.function.name, call.function.arguments)
        # append {"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)} to messages

The exposed tools are remember, job_status, recall, forget, list_by_entity, stats.

Direct API

from memotether import Memotether

with Memotether.from_env("memories.db", embedder="nomic") as mt:
    mt.remember("Long conversation transcript or document...")
    out = mt.recall(
        "what does the user prefer for backend?",
        k=10,
        mode="hybrid",
        tier="both",
    )
    print(out["memories"], out["chunks"])

As an MCP server

Expose Memotether to any MCP-aware client (Claude Desktop, Cursor, opencode, etc.). After pipx install memotether (or via uvx), the memotether-mcp console script starts a stdio MCP server. Six Memotether tools — remember, job_status, recall, forget, list_by_entity, stats — are advertised automatically.

remember over MCP is fire-and-forget: it returns a job_id immediately and runs extraction on a background thread, so the agent isn't blocked waiting on multi-second LLM calls. Use job_status(job_id) only when you need to confirm completion (it's also surfaced in stats()). Direct (non-MCP) callers can still use the synchronous Memotether.remember() method.

The MCP server also serves an agent system prompt under the name memotether-system via prompts/get. The same prompt covers chat, planning, debugging, and coding sessions — there is no mode to set.

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json on macOS) — using uvx so the install is self-contained:

{
  "mcpServers": {
    "memotether": {
      "command": "uvx",
      "args": ["--from", "memotether[local-embed]", "memotether-mcp"],
      "env": {
        "MEMOTETHER_DB_PATH": "/absolute/path/to/memories.db",
        "MEMOTETHER_EMBEDDER": "nomic",
        "OPENROUTER_API_KEY": "...",
        "MEMOTETHER_MODEL": "anthropic/claude-sonnet-4.5",
        "MEMOTETHER_BASE_URL": "https://openrouter.ai/api/v1",
        "MEMOTETHER_MAX_CHUNK_CHARS": "16000"
      }
    }
  }
}

opencode — use the bundled installer; see Install above. memotether install opencode writes the entry into ~/.config/opencode/opencode.json (or ./opencode.json with --project) and, for project scope, drops the agent prompt at AGENTS.md since opencode doesn't yet surface MCP prompts in its UI (sst/opencode#806).

The DB path defaults to ~/.memotether/memories.db if unset. Embedder defaults to nomic (local, ~550 MB on first use); pass --embedder none or MEMOTETHER_EMBEDDER=none to disable vector search.

CLI

Ingest a corpus and run a needle-in-haystack eval:

# Ingest + eval (substring grader)
python scripts/run_eval.py questions.json

# LLM-judge eval comparing tiers
python scripts/run_eval.py questions.json --judge llm --tier memories --out memories.json
python scripts/run_eval.py questions.json --judge llm --tier chunks   --out chunks.json
python scripts/run_eval.py questions.json --judge llm --tier both     --out both.json

Or use the lower-level scripts:

python scripts/run_extraction.py input.pdf --db memories.db
python scripts/query_memories.py memories.db "search query" --mode hybrid

Layout

memotether/
  schema.py      — Pydantic models: AtomicMemory, Chunk, ConversationTurn, MemoryType
  extractor.py   — Extractor class: chunking, parallel extraction, reconciliation,
                    retry/backoff, fallback model
  store.py       — MemoryStore: SQLite + FTS5 + BLOB embeddings; memory + chunk tiers,
                    keyword/fuzzy/vector/hybrid search, supersedes tracking
  embedders.py   — NomicEmbedder (local fastembed), OpenAIEmbedder, resolve_embedder
  tool.py        — Memotether class: unified surface with OpenAI-format tool schemas
  prompts/       — System prompts for extraction and reconciliation (markdown)

scripts/
  run_extraction.py             — ingest a PDF/text into a SQLite store
  query_memories.py             — CLI search
  generate_haystack_questions.py — synthesize needle questions from a long corpus
  run_eval.py                   — end-to-end retrieval eval (substring or LLM judge)

Configuration knobs

Env var	Purpose	Default
OPENROUTER_API_KEY	API key for the extractor / reconciler / synthesizer	required
MEMOTETHER_BASE_URL	OpenAI-compatible API base	required (e.g. OpenRouter)
MEMOTETHER_MODEL	Model id for extraction and (default) reconciliation	required
MEMOTETHER_RECONCILE_MODEL	Model id for the reconciliation pass	falls back to MEMOTETHER_MODEL
MEMOTETHER_FALLBACK_MODEL	Model swap target after repeated rate-limit / transient errors	xiaomi/mimo-v2.5-pro in run_eval.py
MEMOTETHER_MAX_CHUNK_CHARS	Chunking budget per extraction call	required (recommend 16000)
MEMOTETHER_JUDGE_MODEL	Model id for the LLM judge in eval	falls back through RECONCILE_MODEL → MODEL

Embedders:

• embedder="nomic" — local fastembed, nomic-ai/nomic-embed-text-v1.5, 768-dim, ~550MB on first use, Apache 2.0
• embedder="openai" — OpenAI text-embedding-3-small by default
• embedder="nomic:<model>" / "openai:<model>" — override the model
• Pass any Callable[[str], list[float]] or an object with embed_document / embed_query for full custom embedders

The store is dim-aware: swapping embedders mid-life is supported via Memotether.set_embedder(spec, reembed=True), which migrates existing vectors to the new dim.

Reliability

• Per-chunk persistence: extraction commits after each chunk, so a mid-run failure doesn't lose earlier work.
• Retry with exponential backoff: transient API errors (rate limits, timeouts, 5xx) retry up to 6 times with jittered backoff.
• Model fallback: after repeated failures on the primary model, retries swap to a configured fallback.
• Parallel extraction: chunks and reconciliation windows run through a thread pool (default 8 workers).