RLM: Agent Memory Infrastructure

Auto-rollback, episodic memory, and evidence-backed context selection for AI agents.

RLM watches any AI coding agent, catches test failures instantly, rolls back bad edits, and builds memory so the same mistake never happens twice. Works with Cursor, Claude Code, Codex, Aider, or any agent that writes files.

pip install recursive-rlm
rlm watch --test "pytest"

That's it. Your agent writes code. RLM guards the tests.

---

How It Works

Agent edits file → Tests run → Pass? Continue. Fail? Roll back instantly.
                                                     ↓
                                              Store in memory
                                                     ↓
                                         Next time agent touches
                                          this file → warn it:
                                       "this failed 3 times before"

Every rollback creates a verified memory record. After enough verified success/failure trajectories, RLM can train a per-repo selector that changes future context ranking.

---

Quickstart

pip install recursive-rlm
cd your-project

# Watch mode — guard any agent's edits
rlm watch --test "pytest tests/ -q"

# Or run a task directly with rollback protection
rlm run "fix the auth bug" --test "pytest tests/"

# Connect to your IDE
rlm setup cursor    # auto-configures Cursor MCP
rlm setup claude    # auto-configures Claude Desktop MCP

╭──────────────────── Recursive Labs ─────────────────────╮
│ RLM Watch — Active                                      │
│ Repo:  /your/project                                    │
│ Tests: pytest tests/ -q                                 │
╰─────────────────────────────────────────────────────────╯

  Changed: auth.py
  ✗ Newly failing: tests/test_auth.py::test_login
  ↩  Rolling back 1 file(s)...
  ✓ Repo restored. Failure saved to history.
     ~4,000 tokens saved · $0.012 · session total: 1 rollback

---

Features

Core

Command	What it does
rlm watch	Live filesystem guardian. Rolls back any agent's bad edits.
rlm run	Execute a task with rollback protection and memory injection.
rlm demo	Interactive demo: memory warnings + live rollback (no setup needed).
rlm demo --domain config	Same demo, but with JSON schema verification instead of pytest.
rlm status	Show memory, graph, selector, and token savings stats.
rlm report	Weekly summary: rollbacks, patterns learned, cost saved, risky files.

Memory & Intelligence

Command	What it does
rlm index	Index codebase for hybrid BM25 + semantic search.
rlm query "auth flow"	Search codebase and episodic memories.
rlm sync	Cross-repo memory sync — promote recurring failures globally.
rlm dashboard	Local web console: trajectories, call graph, selector weights.

Infrastructure

Command	What it does
rlm init	Scan repo, build dependency graph, prepare runtime.
rlm setup cursor/claude	Auto-configure MCP for your IDE.
rlm serve	Start MCP stdio server (11 tools for agent integration).
rlm benchmark	Run trace-backed real benchmark fixtures.
rlm contribute	Opt in/out of anonymized trajectory sharing.

Advanced

Flag	What it does
--adaptive	Score file risk and adjust context-budget accounting per file.
--max-context-tokens N	Cap token budget for context packing.
--no-history	Disable local history storage.

---

Not Just Code: Domain-Agnostic Verification

RLM isn't locked to pytest. The Verifier interface makes rollback + memory work for any domain:

from RLM.engine.verifiers import JsonSchemaVerifier, HttpVerifier, CompositeVerifier

# Verify config files against a schema
verifier = JsonSchemaVerifier("service.json", schema={
    "required": ["port", "replicas"],
    "properties": {
        "port": {"type": "integer", "minimum": 1024},
        "replicas": {"type": "integer", "minimum": 1},
    }
})

# Verify an API endpoint returns 200
verifier = HttpVerifier("http://localhost:8080/health", expected_status=200)

# Chain multiple verifiers
verifier = CompositeVerifier([pytest_verifier, schema_verifier, http_verifier])

Try it: rlm demo --domain config shows JSON schema verification with the same rollback + memory system.

---

MCP Tools (11 tools for agent integration)

RLM ships a stdio MCP server exposing memory, search, and analysis to any connected agent:

rlm serve   # or: rlm setup cursor

Tool	What it does
get_current_context	Full context: file, trace, blast radius, memories
get_relevant_memories	Query episodic memory for past failures on this path
get_blast_radius	Dependency graph: what breaks if this file changes (Python default; pip install recursive-rlm[graph] for 15+ languages)
get_file_risk_scores	Caution score (0-1) based on past rollbacks + complexity
eval_diff	Analyze current diff: changed symbols, blast radius, past failures, suggested tests
codebase_search	Hybrid BM25 + semantic search over local codebase
write_agent_memory	Agent writes verified outcomes to memory
reindex_codebase	Trigger incremental codebase indexing
get_active_file	Current file, cursor, selection from IDE
get_terminal_trace	Latest command, exit code, trace
get_session_cost	Token spend, cost, savings, and budget remaining for the session

---

The Learning Loop

RLM isn't just a rollback tool. Every verified run can produce a trajectory containing the selected context, the available-but-unselected context, the verifier command, exit code, output hash, repo commit, and diff hash. RLM accumulates these and trains a Context Selector — a per-repo model that estimates which context chunks predict success.

Verified runs → trajectory evidence → selector.partial_fit()
                                       ↓
                              P(success | query, context)
                                       ↓
                         Budget-aware context packing

Out of the box, RLM uses cosine-style ranking. After at least 30 balanced verified runs, the selector activates and returns learned P(success | query, context) scores. The budget solver consumes those probabilities directly when deciding which memories and code chunks fit the context budget.

---

Adaptive Cost / Accuracy Optimization

Not every file deserves the same attention. RLM scores each file by importance and reports an adaptive tier:

rlm watch --test "pytest" --adaptive

File importance = blast_radius × failure_history × complexity

auth.py      → score 0.82 [CRITICAL]  → full context budget
utils/fmt.py → score 0.15 [LIGHT]     → reduced context budget estimate

Tier	Context Budget Multiplier	When
CRITICAL (>= 0.7)	100%	High blast radius, many past failures
STANDARD (>= 0.3)	50%	Normal files
LIGHT (< 0.3)	15%	Leaf files, no dependents, no failure history

This is currently a risk-scoring and context-budget policy, not a published accuracy benchmark. Tier-specific verifier selection is intentionally not claimed until it has a trace-backed evaluation.

---

Evidence Status

The repository enforces the learning loop with tests:

• trajectory save -> selector partial_fit() -> selector score() -> budget solver selection
• selector cold-start and class-balance gates
• wheel install smoke checks and both demos in CI

Benchmark claims should be generated from trace files with rlm benchmark or python -m RLM.experiments.benchmark_suite --mode real .... Synthetic or contract traces must not be presented as real model evidence.

---

Constraint-Aware Planning

RLM includes a ConstraintSpec system inspired by multi-objective RL:

from RLM.engine.constraints import ConstraintSpec

# Define constraints explicitly
spec = ConstraintSpec(
    forbidden_files={"production.env", "deploy.yaml"},
    resource_ceilings={"cost_usd": 1.0, "api_calls": 50},
)

# Or learn constraints from past failures
spec = ConstraintSpec.from_history(history, repo_path=".")
# Automatically marks files with 3+ failures as high-risk

Forbidden actions are structurally masked out before scoring -- an illegal path has probability zero, not a low score.

---

Cross-Repo Memory

Failure patterns that appear across multiple repos are automatically promoted to global scope:

rlm sync                 # auto-promote cross-repo failures
rlm sync --list-global   # see all global memories

A failure pattern learned in repo A surfaces as a warning in repo B across all repos on your machine.

---

Architecture

┌─────────────────────────────────────────────┐
│  CLI (rlm watch / run / demo / report)      │
├─────────────────────────────────────────────┤
│  Engine                                     │
│  ├── WatchLoop (filesystem guardian)        │
│  ├── RollbackLoop (agentic coding loop)     │
│  ├── Verifiers (pytest, schema, http, ...)  │
│  └── ConstraintSpec (Phi — hard masking)    │
├─────────────────────────────────────────────┤
│  Memory                                     │
│  ├── SQLiteHistory (episodic store)         │
│  ├── CodebaseIndexer (BM25 + semantic)      │
│  ├── BudgetSolver (token-aware packing)     │
│  ├── SemanticContextGraph (blast radius)    │
│  └── RL Selector (learned P(success))       │
├─────────────────────────────────────────────┤
│  API                                        │
│  ├── MCP Server (11 tools, stdio)           │
│  ├── ContextOS (unified context layer)      │
│  └── Dashboard (local web console)          │
└─────────────────────────────────────────────┘

---

Privacy

All data stays local. Memory is stored in repo-local .rlm_history.db. Nothing leaves your machine.

rlm contribute --enable   # opt-in to anonymized trajectory sharing
rlm contribute --disable  # opt-out (default)

---

Business Model

The open-source runtime is the wedge. The paid product is shared reliability memory for teams running coding agents every day:

Plan	Buyer	Packaging
Free OSS	Individual developers	Local rollback, memory, MCP, and demos.
Pro	Power users	Hosted sync, benchmark reports, and cross-machine memory.
Team	Engineering teams	Shared repo memory, audit trails, risky-file analytics, and team dashboards.
Enterprise pilot	AI-heavy engineering orgs	Private deployment, trace retention controls, custom benchmark, and integration support.

The first commercial motion is not "sell another coding agent." It is: prove on a customer's repo that RLM reduces repeated failed agent edits and preserves the verified failure/recovery data their existing tools discard.

---

Comparison

Perseus is the cleanest adjacent YC comparison: codebase search for coding agents. RLM's thesis is broader and riskier: codebase search is one input, but the compounding asset is verified rollback memory.

	RLM	Perseus	Claude Code
Primary job	Stop agents repeating failed edits	Give agents citable codebase context	Let a user drive an agent
Episodic failure memory	Yes	Not claimed publicly	No
Auto-rollback on test failure	Yes	No	Manual/user-driven
Learned context selection	Yes, after 30 balanced verified runs	Not claimed publicly	No
Codebase search	Yes, local hybrid search	Yes, hosted multi-hop search	Yes, built in
Exact file/line citations	Yes	Yes	Sometimes
Blast radius analysis	Yes, file + symbol	Yes, graph/search impact claims	Limited
Cross-repo memory	Yes	Not claimed publicly	No
Domain-agnostic verifiers	Yes: pytest, schema, HTTP, composite	No	No
Local-first/offline	Yes	No, hosted product	Yes
Open source runtime	Yes	No	No
Gets better from usage data	Yes, via verified trajectories	Not claimed publicly	No user-owned flywheel

The honest YC framing: Perseus helps agents find the right code before they edit. RLM helps agents recover, remember, and improve after the edit is verified. RLM wins if verified failure/recovery trajectories become the scarce data asset. Perseus wins if search alone captures the budget.

See docs/PERSEUS_COMPARISON.md for the full competitor comparison and docs/BUSINESS_MODEL.md for packaging.

---

YC Readiness

What is strong now:

• Working local wedge: rollback + memory + MCP works without replacing Cursor, Claude Code, Codex, or Aider.
• Test-backed learning path: trajectories feed selector training, and selector probabilities feed budget-aware context packing.
• Clear adjacent-market proof: YC-backed Perseus validates that "context for coding agents" is fundable; RLM's differentiated bet is verified failure memory, not search alone.

What must be proven before calling this 9+/10 externally:

• Public PyPI wheel for the current release works with pip install recursive-rlm && rlm demo.
• Same-base-model SWE-bench or SWE-bench-style ablation shows RLM beats the baseline on task success, repeated-failure reduction, or context efficiency.
• Selector eval shows learned selection beats cosine/BM25 on held-out verified trajectories.
• At least one external user or design partner runs RLM on a real repo.

---

Development

git clone https://github.com/arushsinghal/RLM.git
cd RLM
pip install -e ".[dev]"
pytest RLM/tests/ -q

---

Roadmap

• Now: CLI rollback tool + context selector + MCP server + cross-repo memory
• Next: publish the current wheel, public SWE-bench-style benchmark, team sync, IDE plugin
• Later: RLM-0 foundation model trained on verified failure/recovery trajectories

---

Built by Recursive Labs