memory-os-causal 0.1.1

Causal-graph cross-session memory for LLM agents (memento_v4 import name preserved)

Memento v4

A causal-graph-based cross-session memory system for LLM agents, with a benchmark harness for comparing it against MemPalace on LoCoMo and LongMemEval.

What's in this repo

memento_v4/
├── memento_v4/              ← the memory system (Python package)
│   ├── adapter.py           ← MemoryV4Adapter — the end-to-end pipeline
│   ├── ablation.py          ← AblationFlags: 5 independent feature toggles
│   ├── bm25.py              ← pure-Python Okapi BM25
│   ├── text_utils.py        ← tokenization, stop words, boost helpers
│   ├── llm.py               ← sync LLM client (finalize / classify / conflict / reflect)
│   ├── types.py             ← Conversation / Session / Turn dataclasses
│   └── causal/              ← the underlying causal-memory graph
│       ├── models.py        ← SessionNode, CausalEdge, DomainRecord
│       ├── storage.py       ← pure-file JSON + JSONL + MEMORY.md storage
│       ├── finalize.py      ← LLM session summarisation (1 call/session)
│       └── recall.py        ← rule-based retrieval helpers
├── benchmarks/              ← benchmark harness
│   ├── base.py              ← MemoryAdapter ABC + RecallContext
│   ├── llm.py               ← async httpx client (QA + judge)
│   ├── qa.py                ← shared QA prompt + answerer
│   ├── metrics.py           ← F1 / BLEU-1 / LLM-judge
│   ├── report.py            ← JSON + Markdown report writer
│   ├── dataset_locomo.py    ← LoCoMo loader
│   ├── dataset_longmemeval.py  ← LongMemEval loader
│   ├── mempalace_adapter.py ← MemPalace baseline
│   ├── v4_adapter.py        ← thin wrapper around memento_v4.MemoryV4Adapter
│   ├── run_locomo.py        ← LoCoMo runner
│   └── run_longmemeval.py   ← LongMemEval runner
├── data/                    ← benchmark datasets (copy here after download)
├── scripts/
│   ├── download_data.py     ← fetch LoCoMo + LongMemEval
│   ├── run_locomo.sh        ← one-shot LoCoMo runner
│   └── run_longmemeval.sh   ← one-shot LongMemEval runner (with stratified sampling)
└── results/                 ← benchmark outputs (per-run log + JSON + MD)

---

Quick start (5 minutes)

1. Install

pip install memory-os-causal

Distribution name is memory-os-causal; the import name remains memento_v4.

To run the benchmark harness from a clone of this repo, also install the extra dev/benchmark requirements:

cd memento_v4
pip install -r requirements.txt

Python 3.12+ recommended.

2. (Optional) Download datasets

The data/ directory already ships with LoCoMo + both LongMemEval variants populated. If you need fresh copies:

python scripts/download_data.py              # all
python scripts/download_data.py locomo       # just LoCoMo

3. Run the LoCoMo benchmark

bash scripts/run_locomo.sh

Default: mempalace vs memento_v4 on 2 conversations (~10 min). Output goes to results/locomo/.

4. Run LongMemEval

# Quick: 120 stratified questions covering all 6 types (~45 min)
STRATIFIED=1 MAX_Q=120 bash scripts/run_longmemeval.sh

# Focused: only v4's strongest category (knowledge-update, 78 Qs)
QTYPE=knowledge-update bash scripts/run_longmemeval.sh

# Full: 500 questions (~2 h)
MAX_Q=500 bash scripts/run_longmemeval.sh

---

Configuring the API

By default both scripts call ppapi.ai (a Chinese OpenAI-compatible proxy). Override via env vars:

# OpenRouter
PROVIDER=openrouter OPENROUTER_API_KEY=sk-or-xxx \
  bash scripts/run_locomo.sh

# Direct OpenAI
PROVIDER=openai OPENROUTER_API_KEY=sk-xxx QA_MODEL=gpt-4o-mini \
  bash scripts/run_locomo.sh

# Any custom OpenAI-compatible endpoint (vLLM, LiteLLM, etc.)
OPENROUTER_BASE_URL=http://localhost:8000/v1 OPENROUTER_API_KEY=anything \
QA_MODEL=local-model \
  bash scripts/run_locomo.sh

The script picks model names per provider; override QA_MODEL / JUDGE_MODEL / FINALIZE_MODEL as needed.

---

The 5 ablation flags

v4 adds five independently-toggleable optimizations on top of v3 (vector search + causal chain). Each is motivated by a specific insight from 10 other AI memory systems.

Flag	Inspired by	What it does
classify_memories	Hindsight 4-network	At ingest, classify each finding as fact / preference / lesson with confidence
conflict_detection	EverOS lifecycle + Zep validity	Detect cross-session contradictions, mark superseded sessions, down-rank at recall
reflect_synthesis	Hindsight reflect	At recall, synthesize a concise cross-session answer via LLM (smart-routed — skipped for single-hop factuals)
bm25_scoring	standard hybrid retrieval	Weighted (0.45 vector + 0.55 BM25) fusion for precise term matching
memory_summary	MemPalace lightweight startup	Always prepend the domain-level MEMORY.md hot summary

All five are on by default. Toggle via adapter suffix in --adapters:

# Disable one
ADAPTERS=memento_v4-no_reflect bash scripts/run_locomo.sh

# Disable two
ADAPTERS=memento_v4-no_bm25-no_summary bash scripts/run_locomo.sh

# All off (equivalent to v3 baseline, inside the v4 harness)
ADAPTERS=memento_v4-none bash scripts/run_locomo.sh

# Compare all-on vs one-variant in a single run
ADAPTERS=memento_v4,memento_v4-no_reflect bash scripts/run_locomo.sh

Supported suffixes: no_classify, no_conflict, no_reflect, no_bm25, no_summary, none (all off).

---

Using memento_v4 programmatically (outside the benchmarks)

import asyncio
from pathlib import Path
from memento_v4 import MemoryV4Adapter, AblationFlags
from memento_v4.types import Conversation, Session, Turn

async def main():
    conv = Conversation(
        conv_id=1,
        speaker_a="user",
        speaker_b="assistant",
        sessions=[
            Session(
                session_num=1,
                date_time="2026-04-01",
                turns=[
                    Turn(speaker="user", text="Let's pick a database."),
                    Turn(speaker="assistant", text="How about PostgreSQL?"),
                    Turn(speaker="user", text="Sounds good, let's go with PG."),
                ],
            ),
            Session(
                session_num=2,
                date_time="2026-04-15",
                turns=[
                    Turn(speaker="user", text="PG latency p99 > 5s, we need to move."),
                    Turn(speaker="assistant", text="Let's migrate to MongoDB."),
                ],
            ),
        ],
    )

    adapter = MemoryV4Adapter(
        memory_root=Path("./my_memory"),
        model="gemini-3-flash-preview",
        ablation=AblationFlags(),  # all 5 features on
    )
    await adapter.setup()
    await adapter.ingest(conv, conv_id="demo")

    ctx = await adapter.recall("Why did we switch databases?", "demo")
    print(ctx.raw_text)
    await adapter.teardown()

asyncio.run(main())

---

Expected runtimes (reference)

Benchmark	Config	mempalace	memento_v4
LoCoMo (2 convs)	304 QAs, concurrency=5	~12 min	~16 min
LongMemEval-oracle	120 stratified	~14 min	~94 min
LongMemEval-oracle	500 questions	~60 min	~6 h
LongMemEval-S	100 Qs, 53-sess haystack	~3 h	~10 h

v4 is ~6× slower than mempalace due to the extra LLM calls (classify_memories + conflict_detection per session during ingest, reflect_synthesis per query during recall). Turn off optimizations you don't need with the no_* adapter suffixes.

---

Troubleshooting

ModuleNotFoundError: No module named 'chromadb' — the scripts default to the locomo conda env at /Users/huichi/miniconda3/envs/locomo. Override with PYTHON=/path/to/your/python, or install deps into your active env: pip install -r requirements.txt.

401 / Missing Authentication header — OPENROUTER_API_KEY is unset or the wrong key for the current PROVIDER. See the API configuration section.

Error code: 503 — the upstream provider rate-limited. The runners retry 3× with exponential backoff; if you see many "GAVE UP" messages, lower CONCURRENCY or switch to a different provider.

response.text is None / AttributeError — a known issue with thinking models returning content: null when the reasoning budget is exhausted. Already handled defensively across qa.py, metrics.py, and llm.py.

---

License

MIT (TBD — see original memento-s / cc-mini licenses).