DSPM — Dynamic Semantic Patch Memory

by Dhruv Dubey

Compress multi-turn LLM conversations while keeping every constraint, decision and revision in the memory store.

At a 250-token budget, DSPM reduced tokens by about 83% in our benchmark suite while retaining 100% of critical patches. Under very small budgets, rendered lines are trimmed hard and some facts may be stored but not shown. See Known Limitations.

Who it's for: agents and chatbots whose long conversations must not lose a constraint, a decision, or a revised value (support bots, coding agents, assistants that remember a user's hard rules across sessions).

Who it's not for: summarizing or answering questions over documents. See Known Limitations.

See It Work

A payment-API conversation where a value changes mid-chat. After compression, only the current value remains, and the hard rules are intact:

from openai import OpenAI
from dspm import DSPMMemory

llm = OpenAI(api_key="sk-...")   # any OpenAI-compatible endpoint works

# budget=60 so compression is visible even in this small example;
# use 200-300 for real conversations
memory = DSPMMemory(budget=60, llm_client=llm, model="gpt-4o-mini")

memory.add_turn("user", "Building a payment API. Hard rules: PCI-DSS compliant, max fee 0.5%, deadline Friday.")
memory.add_turn("assistant", "PCI-DSS needs tokenized card storage and quarterly ASV scans. Tracking the 0.5% fee cap.")
memory.add_turn("user", "Webhook timeout must be 30 seconds.")
memory.add_turn("assistant", "Webhook timeout set to 30 seconds with automatic retries.")
memory.add_turn("user", "Change the webhook timeout to 10 seconds instead — 30 is too slow.")
memory.add_turn("assistant", "Updated: webhook timeout is now 10 seconds; the 30s setting is superseded.")

print(memory.get_context(query="What are the hard requirements?"))
print(memory.stats)

Example output (recorded with v0.1.17; exact wording varies by extraction model):

[DEC] Track the 0.5% fee cap.
[CON] PCI-DSS requires tokenized card storage and ASV
[DEC] webhook timeout updated to 10 seconds
[CON] PCI-DSS compliant, max fee 0.5%, deadline Friday
{'turns': 6, 'total_patches': 4, 'critical_total': 4, 'critical_selected': 4, 'crr': 100, 'budget': 60, 'raw_tokens': 61, 'context_tokens': 56, 'trr': 9, ...}

Only the 10-second entry remains; the 30-second setting is superseded. At even tinier budgets the revision story is kept in minimal form, e.g. 10 (was 30).

Try to break it: give it a conversation where a value changes several times under a very small budget, and open an issue if a constraint or revision is lost.

The Problem

Long conversations eat your context window. Naive truncation drops the constraint from turn 3 that the entire system depends on. DSPM is built so that does not happen.

How It Works

DSPM converts each conversation turn into typed semantic patches (constraint, decision, code, equation, entity, structure) and compresses them under a fixed token budget. Critical patches (constraints and decisions) are structurally protected in the store and are selected first when the context is rendered.

Raw conversation (452 tokens, 18 turns)
        ↓
Semantic extraction → 28 patches, 18 critical
        ↓
6-stage compression pipeline
        ↓
Compressed context (249 tokens) — 100% of criticals selected

The 6-stage pipeline: dedup → slot fusion → delta encoding → causal pruning → utility scoring → shadow selection. (A 7th stage, adaptive per-type budgeting, is designed but not wired in this release.)

Install

pip install dspm-memory

Requires Python 3.9+. Works with any OpenAI-compatible LLM provider.

Optional: semantic ranking

pip install "dspm-memory[semantic]"

Adds sentence-transformer-based query alignment for better non-critical patch ranking (downloads PyTorch and a small embedding model on first use). Without it, DSPM falls back to a neutral alignment score.

Supported Providers

# OpenAI
llm = OpenAI(api_key="sk-...")

# Groq (free tier available)
llm = OpenAI(base_url="https://api.groq.com/openai/v1", api_key="gsk_...")

# Together AI
llm = OpenAI(base_url="https://api.together.xyz/v1", api_key="...")

# Ollama (local, no key needed)
llm = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

What's New in 0.1.18

• feedback_context now defaults to False. In earlier versions the last rendered (budget-trimmed) context was fed back into the extraction prompt of the next turn. In our diagnostic runs the model sometimes re-extracted those trimmed lines as new patches. In one run a fused fragment (Connect retry updated to 120 (was 300)) superseded two real facts. Pass feedback_context=True to restore the old behavior.
• LLM telemetry in memory.stats: llm_calls, llm_empty, llm_errors, llm_retries, llm_length_finish, llm_parse_failures, llm_fallbacks, plus llm_echo_drops and llm_unattributed_drops when the echo guard is used. A turn whose extraction fails silently (empty or unparseable output) is now counted in llm_fallbacks.
• strict_llm=True: raises instead of silently producing a 0-patch turn when the extraction output is empty, truncated, or not parseable.
• dump_store(include_history=False): returns every stored patch (text, type, criticality, turn, whether it was in the last rendered context), so "forgotten" can be told apart from "stored but not rendered". With include_history=True it also returns the supersession and dedup journals.
• Opt-in, experimental: echo_guard=True drops extracted patches that look copied from the fed-back context (only relevant with feedback_context=True); feedback_fence=True wraps the fed-back context in do-not-extract markers; protect_bound_wording=True stops a plain restatement from replacing a stored bound.
• Reasoning-model safety: reasoning_param_mode ("top", "extra_body", "off") controls where reasoning_effort is sent for gpt-oss models; max_tokens_floor enforces a minimum output budget for extraction calls.

See CHANGELOG.md for details.

Persistence: Memory Across Chats and Sessions

Save the notebook when a chat ends, load it when the next one starts. Chat 2 remembers Chat 1, and revisions supersede old values across sessions:

# Chat 1 — Monday
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
memory.add_turn("user", "Building a budgeting app. Hard rules: must work offline, deadline Oct 20.")
# ... chat ...
memory.save("user_dhruv.json")

# Chat 2 — Thursday, new process, fresh start
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
memory.load("user_dhruv.json")
memory.get_context(query="What were my hard rules?")
# → [CON] must work offline
# → [CON] deadline Oct 20        ← recalled from Chat 1, zero API cost

# Revisions work across chats too:
memory.add_turn("user", "The deadline moved to November 5.")
# → Oct 20 is superseded. November 5 replaces it.

Save files are portable JSON, written atomically (a crash mid-save can't corrupt the notebook), and merge-on-load means saved revisions supersede stale values. One file per user = each person's long-term memory.

What is saved: the patches, turn count and token counters. Not saved: the supersession/dedup journals from dump_store(include_history=True), the last rendered context, and LLM telemetry. These start empty after load().

The Guarantee

What DSPM is built to guarantee, and where it stops:

• In the store: [CON] and [DEC] patches are never dropped by deduplication, fusion, or pruning.
• At render time: criticals are selected first. Under budget pressure their payloads are trimmed numbers-first, with units, negations and comparators protected where the budget allows.
• Revisions: when a constraint is revised mid-conversation (e.g. TTL 60s → 300s), the new value supersedes the old, including cross-type revisions, terse patches, and full revision chains. Under a starvation budget the revision story survives in minimal form: 10 (was 30).
• Last resort: a critical is dropped from the rendered context only when every critical is already at its word floor and the budget still cannot hold them. It stays in the store.
• Extraction failures: transient API failures (rate limits, 5xx) and empty responses from reasoning models are retried automatically with backoff (three attempts). gpt-oss models run extraction at low reasoning effort by default. Use strict_llm=True and memory.stats to see failures.
• Ablation result: removing the shadow-selection mechanism collapses CRR from 100% to 37.9%, isolating the retention behavior to a single identifiable component.

Important limits of this guarantee:

• It covers what the extraction model captures as a constraint or decision. If the extractor misses or misclassifies something, DSPM cannot protect it.
• "CRR" is the share of critical patches selected into the context, not the share of facts a reader can recover from the text. At small budgets a selected line can be trimmed so far that it loses its unit, operator or subject (for example must not exceed 90 or 120 (was 300)).
• In the 0.1.18 diagnostic runs below, facts were present in the store while the rendered context under-delivered at small budgets.

Results

Benchmark suite (v0.1.17)

Tested across 7 domains × 40 turns each:

CRR = Critical Retention Rate (critical patches selected into the context). TRR = Token Reduction Ratio.

Provenance: measured on 7 hand-authored 40-turn technical dialogues (API design, ML ops, IoT, security IR, supply chain, clinical workflow, project management); extraction and judging by GLM via OpenRouter. The same model family performed both extraction and judging, and the benchmark has not been independently replicated. Numbers vary by extraction model and domain; check your own with memory.stats.

Diagnostic runs (v0.1.18, small sample)

One synthetic 48-message alerting-pipeline conversation (12 core messages, 36 filler messages, one revision, one bound), openai/gpt-oss-20b on Groq, two runs per setting. "Facts delivered" counts how many of 8 ground-truth facts a fixed substring scorer finds in the context rendered after message 48.

What we observed:

• The memory store held all core facts in almost every run; the loss was mostly at render time under a small budget.
• With feedback on, results varied between identical runs (7/8 vs 3/8 at budget 120); with it off they did not.
• echo_guard blocked the fact-deleting case (and caught 5 echoes in its one live run), but it did not fix budget crowding: filler decisions such as a pod-memory limit are also extracted as critical and compete for slots.
• Across several hundred live extraction calls, none returned empty or failed.

This is one conversation, a few runs per setting, one model, and a substring scorer. Treat it as a diagnosis, not a benchmark.

Comparison & Performance Harnesses

experiments/benchmark_compare.py measures DSPM against a same-budget rolling summary on fixed scenarios with shared gold checks. The offline mock (--mode mock) prints ground-truth retention per scenario; --mode live runs a real extractor, a true rolling summary, and an LLM judge over a dialogues file you supply (--dialogues), reporting mean ± std across dialogues.

python experiments/benchmark_compare.py --mode mock
python experiments/benchmark_compare.py --mode live --base-url https://api.groq.com/openai/v1 \
    --model openai/gpt-oss-20b --dialogues dialogues.json

Mock-mode reference: DSPM 100% mean retention vs rolling-window 70% at budget 250. These are pipeline-check numbers, not extraction quality; run --mode live for that.

experiments/bench_kv_cache.py answers whether a stable patch context keeps more of the prompt prefix byte-stable than a same-budget raw last-K window (reference: 85.5% vs 70.7% over 40 turns at budget 250). experiments/bench_kv_live.py measures cached-token reuse on a real endpoint. Neither harness yet accounts for extraction latency. See experiments/README.md.

python experiments/bench_kv_cache.py
python experiments/bench_kv_cache.py --sweep
python experiments/bench_kv_live.py

Diagnostic scripts (v0.1.18)

scripts/diag_48.py runs the 48-message diagnostic at budgets 120 and 60 and saves the rendered context, store dump and stats at messages 12/24/36/48; scripts/analyze_diag.py classifies each ground-truth fact as in the store, in the context, both, or neither. Both need GROQ_API_KEY. Flags: --no-feedback, --echo-guard, --fence, --out-dir.

Repository Layout

.
├── src/dspm/
│   ├── config.py
│   ├── patch.py
│   ├── extractor.py
│   ├── memory.py
│   ├── engine.py
│   └── persistence.py
├── tests/                        # 150 offline tests (pytest)
├── scripts/                      # live smoke gates and diagnostics (need GROQ_API_KEY)
│   ├── smoke_conversation.py
│   ├── diag_smoke_criticals.py
│   ├── diag_48.py
│   ├── analyze_diag.py
│   └── ...
├── experiments/
│   ├── benchmark_compare.py
│   ├── bench_kv_cache.py
│   ├── bench_kv_live.py
│   └── results/
├── examples/
│   └── basic_usage.py
├── pyproject.toml
├── CHANGELOG.md
├── LICENSE
└── README.md

Run the offline tests with pytest tests -q. Do not run a bare pytest from the repo root: some scripts under scripts/ call a live API when collected.

Known Limitations

• Small budgets under-deliver rendered facts. At budget 60 in our diagnostic conversation, only 2–3 of 8 facts appeared in the rendered context even though the store held them. The renderer trims selected lines to a word floor, so units, operators or subjects can be lost (for example must not exceed 90 or 90 days instead of must not exceed 90 days).
• Filler can crowd out core facts. Routine status messages (such as "pod limit raised to 6Gi") can be extracted as decisions, which are critical, and then compete with older hard rules for slots. Selection has no notion of importance beyond recency and alignment.
• Duplicate restatements: when the same fact is stated as both a constraint and a decision, the pair is not merged and both consume budget (observed as [CON] within 2 plus [DEC] 2-second delivery).
• Non-numeric facts can be missed: identity facts without numbers (for example the name of a plan) were extracted in some runs and not others, and were never rendered.
• Feedback context (feedback_context=True) can cause the extractor to re-extract trimmed context lines; see What's New. echo_guard reduces the damage but is experimental: its thresholds were tuned on a single failure, and it can drop a genuine update that shares no wording with its turn.
• Persistence: load() restores patches but not the history journals, the last rendered context, or telemetry.
• Numberless obligation echoes: when a requirement is restated without any numeral (e.g. "all payments must be PCI-DSS compliant" restated as "PCI-DSS compliance"), no collapse rule can pin the pair, because the restatement rules are number-anchored. Structured slot identity is planned for a later release.
• Polarity under starvation: negations and comparators are protected down to the 2-word floor, but at the floor a comparator phrase can degrade to its bare value.
• Extraction depends on the LLM: quality varies by model and domain, and a constraint the extractor misses is not protected. Extraction is not fully deterministic even at temperature 0.
• Document QA: DSPM is designed for conversations. In head-to-head document QA tests it lost to a query-aware compressor. Use it for dialogue memory, not for compressing papers or contracts.
• Benchmark scope: 7 hand-authored dialogues plus one synthetic diagnostic conversation. Treat the numbers as a baseline, not a third-party evaluation.
• Still early: version 0.1.x, Beta. APIs may change.

Document Mode: Dense Extraction

# automatic — fact-dense turns route through the dense funnel:
memory.add_turn("user", "p95 latency 200ms, fee 0.5%, timeout 30 seconds, budget 250 tokens")

# documents:
n_chunks = memory.add_document(open("paper.txt", encoding="utf-8").read())

# force dense mode:
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini", dense=True)

API Reference

DSPMMemory(budget, llm_client, model, ...)

The budget can be changed at any time:

memory.budget = 100
memory.set_budget(100)   # explicit alternative — same effect

Methods

Properties

Paper

DSPM: A Critical-Retention Approach to Long-Context Memory Compression for LLM Conversations
Dhruv Dubey, 2026
Zenodo: 10.5281/zenodo.19438636

Feedback

Found a case where a constraint or revision is lost? Please open an issue with the conversation and the budget you used.

License

MIT © 2026 Dhruv Dubey