memvault-ai 0.1.0

Open source, local-first persistent memory system for LLMs

⚡ MemVault

Open source, local-first memory system for Claude and MCP workflows.

Zero data leaves your machine. Ever.

---

The Problem

LLMs are stateless mathematical functions — y = f(x, θ) — that forget everything between sessions.

Metric	Value
6-month AI conversation history	~19.5 million tokens
Largest context window (Claude)	~200k tokens
MemVault wake-up cost	~350 tokens
Annual cost estimate	~$10/yr
Data leaving your machine	Zero

Quick Start

# Install
pip install memvault

# Initialize
memvault init

# For manual backfill, mine a project's Claude sessions first
memvault mine-convos ~/.claude/projects/-Users-you-my-project --project my-project --track auth

# Recall project decisions from session memory
memvault recall "What database did we choose for the auth service?" --project my-project --track auth

# Index project code/docs separately when needed
memvault index-project /path/to/repo --project my-project --track auth

# Search indexed project files separately
memvault search-project "database migrations" --project my-project --track auth

# Generate wake-up context for a new AI session
memvault wake-up

Recommended Beta Workflow

MemVault is most useful when we keep session memory and project indexing separate.

• Use mine-convos and recall for session recall, decisions, and prior discussions.
• Use index-project and search-project only when you explicitly want code/docs indexing.
• Scope search to a project whenever possible.
• Use --track to narrow within a project, for example auth, infra, or onboarding.
• Avoid broad mixed ingestion as the default beta workflow.
• For Claude, prefer the automatic hook flow for new sessions and use mine-convos for backfills.

For decision-style questions, MemVault now favors conversational memory before raw file text.

Automatic Claude Setup

For a one-time Claude installation with automatic session-start injection, automatic session-end mining, and MCP tool wiring:

memvault init
memvault install --client claude

After that, Claude will:

• inject MemVault context automatically at session start
• mine the finished transcript automatically at session end
• infer project and track from the working directory and transcript when possible
• expose MemVault MCP tools during the session

Recommended Claude pattern:

1. Install once with memvault install --client claude
2. Work normally in Claude inside a project folder such as /Users/you/keystone/backend/auth
3. Let the stop hook file the session into the keystone project and auth track automatically
4. Use memvault recall --project keystone --track auth only when you want to inspect what got stored manually

MCP Integration (Cursor, VS Code, etc.)

Add to your MCP settings:

{
  "mcpServers": {
    "memvault": {
      "command": "memvault",
      "args": ["serve"]
    }
  }
}

The AI assistant automatically gets 20 MCP tools for search, context retrieval, duplicate checks, memory management, knowledge graph access, and contradiction workflows.

If your MCP client can send its current working directory, MemVault can now infer:

• project from the active repo path
• track from the current sub-area such as auth, infra, or onboarding
• the best retrieval lane automatically, with a softer fallback if the first lane is empty

Architecture

4-Layer Memory Stack

┌──────────────────────────────────────────────────┐
│  L0  Identity               ~50 tokens   ALWAYS  │
├──────────────────────────────────────────────────┤
│  L1  Critical Facts         ~300 tokens  ALWAYS  │
├──────────────────────────────────────────────────┤
│  L2  Project Context        ~2,000 tokens ON-DEM │
├──────────────────────────────────────────────────┤
│  L3  Deep Search            ~5,000 tokens ON-DEM │
└──────────────────────────────────────────────────┘

Wake-up loads L0+L1 only (~350 tokens). Deeper context is fetched only when needed through MCP tool calls.

Memory Types

Type	Store	Examples
Episodic	ChromaDB	Past events, sessions, decisions
Semantic	SQLite KG	Facts about user, projects, entities
Procedural	SQLite	Preferences, coding style, guardrails

Retrieval Strategy

MemVault keeps two retrieval lanes:

• Session memory for decisions, prior discussions, and corrections
• Project memory for code/docs context that you explicitly index

Each lane can be narrowed with:

• project for the overall codebase or initiative
• track for a narrower thread inside that project, such as auth, infra, or onboarding

For beta, the intended flow is:

1. Let Claude auto-mine new sessions through hooks
2. Backfill older conversations with mine-convos when needed
3. Recall within a project and optional track
4. Index project files separately only when needed
5. Pull only the relevant memories needed for the current question

Hybrid Search Pipeline

Query ─┬─► BM25 Keyword Search ──┐
       │                          ├─► RRF Fusion (k=60) ─► Scoring ─► [Reranker] ─► Results
       └─► ChromaDB Vectors ─────┘

Scoring formula: final = relevance×0.40 + confidence×0.35 + recency×0.25

Intelligence Features

• Contradiction Detection — Rule-based + optional NLI (DeBERTa-v3-base). Every fact is checked against existing knowledge.
• Entity Detection — 80+ technologies, concepts, and person names auto-detected and registered in the Knowledge Graph.
• Memory Decay — Time-based confidence decay (soft at 180 days, hard at 365 days). Keeps memory fresh.
• Extractive Summarization — Decision-focused sentence scoring. No external LLM required.

Benchmark Results

Current benchmark results from local runs:

Config:      Vector-only
Queries:     15
R@1:         86.7%
R@3:         100.0%
R@5:         100.0%
R@10:        100.0%
MRR:         0.9333
Avg Latency: 384.3ms

Config:      BM25 + Vector + RRF (Hybrid)
Queries:     15
R@1:         86.7%
R@3:         93.3%
R@5:         100.0%
R@10:        100.0%
MRR:         0.9167
Avg Latency: 309.5ms

Config:      BM25 + Vector + RRF + CrossEncoder
Queries:     15
R@1:         86.7%
R@3:         93.3%
R@5:         100.0%
R@10:        100.0%
MRR:         0.9167
Avg Latency: 739.3ms

Detailed report: benchmarks/REAL_BENCHMARK_REPORT.md

Run benchmarks: python benchmarks/longmemeval_bench.py

Tech Stack

Component	Technology
Language	Python 3.9+
Package Manager	uv
Vector Store	ChromaDB ≥0.4.0 (embedded, no server)
Knowledge Graph	SQLite temporal entity-relationship triples
Embeddings	all-MiniLM-L6-v2 (local, 80MB)
Keyword Search	rank-bm25 with RRF hybrid fusion
Re-ranking	ms-marco-MiniLM-L-6-v2 (optional, local)
Contradiction	DeBERTa-v3-base NLI (optional, local)
MCP Server	mcp Python SDK
CLI	Typer + Rich

CLI Commands

# Core
memvault init [--dir DIR]           # Initialize MemVault
memvault install [--client claude]  # One-time client integration install
memvault status                     # System status
memvault mine <dir> [--mode convos] [--project P]  # Beta default: conversation-first ingestion
memvault mine-convos <dir> [--project P] [--track T]   # Session memory only
memvault index-project <dir> [--project P] [--track T] # Project files/docs only
memvault search "query" [--scope auto] [--cross]       # Hybrid search with lane control
memvault recall "query" [--project P] [--track T]      # Session-first recall
memvault search-project "query" [--project P] [--track T] # Project/document search
memvault wake-up [--raw]            # L0+L1 context

# Memory Ops
memvault health                     # Memory health dashboard
memvault explain "query" [--project P] [--track T]      # Why MemVault would answer this
memvault inspect-memory <id>        # Show stored memory text + metadata
memvault edit-memory <id> [--text]  # Correct stored memory text/metadata
memvault save-memory "text" [--kind decision] [--project P] [--track T] # Durable manual filing
memvault tracks --project P         # Review known tracks within a project
memvault dashboard [--project P]    # Pilot-friendly project/track dashboard
memvault review-stale [--project P] # Review stale memories that need cleanup
memvault contradictions [-p PROJ]   # List contradictions
memvault resolve <id> [--keep new]  # Resolve contradiction
memvault decay [--dry-run]          # Apply memory decay
memvault forget [concept|--project] # Selective deletion
memvault rebuild-layers             # Refresh derived L0/L1 context inputs

# Server
memvault install-claude             # Claude-specific installer
memvault serve                      # Start MCP server (stdio)

MCP Tools

Tool	Description
memvault_status	System status and memory counts
memvault_search	Hybrid search across all stores
memvault_get_context	Wake-up context (L0+L1, ~350 tokens)
memvault_get_project_context	Project-specific L2 context
memvault_list_projects	List tracked projects
memvault_list_entities	List KG entities
memvault_check_duplicate	Check whether a memory likely already exists
memvault_explain_answer	Explain which stored memories support an answer
memvault_get_memory	Inspect one stored memory by ID
memvault_add_memory	Add a memory directly through MCP
memvault_edit_memory	Edit a stored memory and its scope/metadata
memvault_delete_memory	Delete a memory by ID
memvault_forget	Forget by concept, project, or entity
memvault_update_confidence	Update a memory confidence score
memvault_detect_entities	Detect + register entities from text
memvault_kg_query	Query the knowledge graph
memvault_kg_add	Add a knowledge graph triple
memvault_kg_invalidate	Invalidate a knowledge graph triple
memvault_kg_timeline	Show an entity timeline
memvault_kg_cross_project	Query KG facts across projects
memvault_health	Return memory health metrics
memvault_contradictions	List detected contradictions
memvault_resolve	Resolve a contradiction

Optional Dependencies

# Cross-encoder re-ranking (improves precision)
pip install memvault[reranker]

# NLI contradiction detection (semantic)
pip install memvault[nli]

# Everything
pip install memvault[reranker,nli]

Development

# Clone and install
git clone https://github.com/memvault/memvault.git
cd memvault
uv pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Run benchmarks
python benchmarks/longmemeval_bench.py --all -v

# Lint
ruff check memvault/ tests/

# Type check
mypy memvault/

Project Structure

memvault/
├── memvault/
│   ├── __init__.py            # Package exports
│   ├── cli.py                 # Typer CLI
│   ├── config.py              # Configuration + constants
│   ├── knowledge_graph.py     # SQLite temporal KG
│   ├── user_model.py          # User preferences + memory metadata
│   ├── vectorstore.py         # ChromaDB wrapper
│   ├── convo_parser.py        # Multi-format parser
│   ├── session_scorer.py      # Session quality scoring
│   ├── ingestor.py            # Chunking + embedding pipeline
│   ├── searcher.py            # Hybrid search (BM25+Vector+RRF)
│   ├── reranker.py            # Cross-encoder re-ranking
│   ├── layers.py              # L0-L3 memory stack
│   ├── context_builder.py     # Dynamic token budget packing
│   ├── fact_checker.py        # NLI contradiction detection
│   ├── entity_detector.py     # Entity detection + registration
│   ├── memory_pipeline.py     # End-to-end orchestrator
│   ├── summarizer.py          # Extractive summarization
│   ├── claude_integration.py  # Claude lifecycle + MCP installer
│   ├── mcp_server.py          # MCP server tools for search, save, inspect, and KG access
│   └── onboarding.py          # Guided setup flow
├── tests/                     # 329 tests
├── benchmarks/                # LongMemEval benchmark suite
├── hooks/                     # Legacy shell hook examples
└── pyproject.toml

How It Works

1. Mine Conversations — Parse sessions first, score them, chunk them, embed them, and register entities in the KG.
2. Index Projects — Index code/docs separately when you want implementation lookup.
3. Recall — Decision queries favor session memory before file text.
4. Inspect / Edit — Use explain, inspect-memory, and edit-memory to debug or correct what MemVault stored.
5. Search Projects — Implementation queries can target indexed project files explicitly.
6. Wake-Up — Load ~350 tokens of identity + critical facts at session start. No expensive retrieval.
7. On-Demand — When the AI needs deeper context, MCP tools fetch scoped memory on the fly.
8. Integrity — New facts are checked against existing knowledge for contradictions. Stale memories decay over time.

Claude Lifecycle

memvault install --client claude writes three Claude hooks plus the MemVault MCP server entry into ~/.claude/settings.json:

• SessionStart runs python -m memvault hook session-start and injects wake-up context automatically.
• Stop runs python -m memvault hook stop and mines the Claude session transcript automatically.
• PreCompact runs python -m memvault hook precompact and applies memory decay before compaction.

The installer is safe to re-run: it updates existing MemVault hooks, preserves unrelated Claude settings, and creates a timestamped backup before modifying an existing settings file.

Beta Notes

• Conversation-first mining is the recommended default for Claude workflows.
• Project indexing is optional and should be used intentionally.
• Dependency folders and generated content are excluded from raw file mining by default.
• The cross-encoder reranker is optional and slower; the hybrid default is usually the best latency/quality tradeoff.
• The intended workflow is mine-convos + recall first, then index-project + search-project when code/docs lookup is needed.

Pilot Workflow

For an internal pilot, the most practical workflow is:

1. Install once for Claude:

   memvault init
   memvault install --client claude
   

2. Work normally in Claude inside a repo or sub-area like keystone/backend/auth
3. Let Claude auto-mine sessions on exit
4. Ask recall questions naturally in a new Claude session
5. Use the inspection tools when needed:

   memvault explain "What database did we choose for keystone auth?" --project keystone --track auth
   memvault inspect-memory <memory-id>
   memvault edit-memory <memory-id> --confidence 0.95
   memvault dashboard
   memvault review-stale --project keystone
   

License

MIT — see LICENSE.