byheart

Conversation memory that knows the dialogue by heart.

Most memory layers take notes about the conversation: they extract facts, rewrite them, and discard the rest. byheart remembers the conversation verbatim — what reaches the model is the dialogue's own words, selected, never paraphrased. Nothing is lost at ingest, because nothing is extracted at ingest.

~400 lines of code      stdlib only: math, re, datetime
0 model calls           at ingest and at composition
0 infrastructure        no server, no vector store, no embeddings

Install

pip install yait-byheart

Use

from yait_byheart import Memory

mem = Memory(budget=20_000)          # characters, ≈ 5 000 tokens

mem.append("user", "we decided on 55 euro a month", when="8 May, 2023")
mem.append("assistant", "Noted — 55/month, annual at a discount.")
# ... hundreds of turns later ...

result = mem.compose("what did we decide about pricing?")
result["messages"]      # ready for any provider, in the universal format
result["stable_upto"]   # cache breakpoint: everything up to here repeats
                        # verbatim between turns — mark it and pay a tenth

compose returns a plain messages array, so it drops into any framework and requires none. Persistence is mem.to_dict() / Memory.from_dict(...) — plain JSON-safe data; where it lives is your business.

How it works — four rules

1. Below the budget, send everything. A full history on a warm prompt cache is cheaper than any clever assembly.
2. Above it, select whole topic stretches by relevance. Everything competes, the newest turns included — recency holds its place by being on topic, not by reservation. Grouping turns into stretches is worth +0.10 evidence recall over single turns; the boundaries come from a character counter, because an LLM segmenter measured no better (±0.5 pp, 0.3 σ) and cost twenty model calls per conversation.
3. Keep the selection still. The selection changes only when the subject genuinely moves, so the prompt prefix stays byte-identical between turns and the provider's cache holds. Measured live: −90.8 % of the per-turn bill, with every cache miss falling exactly on a rebuild. This is the rule competitors cannot copy — a selection recomputed per query has no stable prefix to cache, by construction.
4. Outside knowledge arrives last, and only when it beats the conversation itself. A comparison, not a threshold: nothing to tune, and it holds at 92 % junk rejected / 89 % needed admitted on the free retriever.

Plus one repair that is not a rule: relative dates are resolved by arithmetic at ingest — "I ran the race last Saturday" becomes "… last Saturday [20 May 2023]", annotation beside the words, never instead of them. Worth +11 points on time-anchored questions.

Documentation

Measured

Everything below is reproducible from eval/; the raw traces, including every model answer and every judge verdict, are in eval/results/.

The five points behind Mem0 are the price of extraction never happening. The threefold running-cost advantage is the price extraction keeps charging them: two model calls per ingested exchange, and a per-query context no cache can hold.

Limits, honestly

• English-leaning retrieval. BM25 with a light English stemmer. It degrades gracefully on paraphrase (measured), but across languages lexical match fails entirely — the Russian word for "memory" shares no character with it. retriever accepts anything with fit/rank if you need dense retrieval — a cosine/BM25 hybrid was built and measured here, and netted zero on a monolingual corpus, which is why none ships.
• The cache economics assume an active conversation. The provider cache refreshes on read but expires in minutes of silence; a cold return pays full price once.
• Benchmarked on one model pair (Haiku answering, Sonnet judging), 385 of 1 536 LoCoMo questions, stepped with pre-registered stopping rules.
• Not thread-safe. One conversation, one Memory, one thread.

Repository

yait_byheart/  the package — memory.py, _time.py, ~400 lines of code
tests/         47 tests, including the live-usage path no bench covers
eval/          every benchmark, the measurement protocol, raw result traces