DSPM — memory for LLM conversations that keeps your constraints

Dynamic Semantic Patch Memory, by Dhruv Dubey · Python package: dspm-memory · Repo: Kernl

pip install dspm-memory

Long chats drop the rule from turn 3 that everything else depends on. DSPM turns each turn into typed patches, protects constraints, decisions and revised values, and renders them under a fixed token budget. Training-free: no fine-tuning, just prompts and Python rules. Works with any OpenAI-compatible endpoint.

Headline numbers (v0.2.1, verified on live data)

24-turn fintech conversation, 648 raw tokens, 3 revisions tracked (JWT 15→30min, S3 1h→15min, CPU 70→60%), real Groq LLM:

CRR = Critical Retention Rate. TRR = Token Reduction Rate (conversation-based). CRR = 100% is a structural guarantee, verified across budgets 59 to 5000 tokens.

Offline benchmark (v0.1.17, 7 hand-authored 40-turn dialogues, same model for extraction and judging):

Recognition: Poster accepted at CAICS 2026 (Indo-US Workshop on AI & Cybersecurity), organized by IISc Bengaluru and Argonne National Laboratory, funded by the U.S. Department of State.

Savings depend on conversation length. Short conversations compress very little. Recommended budget: 250–400 for best readability + compression balance.

Who it's for: agents and chatbots whose long conversations must not lose a constraint, a decision, or a revised value (support bots, coding agents, assistants that remember a user's hard rules across sessions).

Who it's not for: summarizing or answering questions over documents. See Known Limitations.

Quickstart

from openai import OpenAI
from dspm import DSPMMemory

llm = OpenAI(api_key="sk-...")   # any OpenAI-compatible endpoint
mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")

mem.add_turn("user", "Always return JSON. Use POST /v1/items. Max fee 0.5%.")
mem.add_turn("assistant", "Webhook timeout set to 30 seconds.")

# Later: the webhook timeout is revised
mem.add_turn("user", "Webhook timeout updated to 10 seconds.")

print(mem.get_context(query="What must the contract remember?"))
print(mem.stats)

Illustrative output (exact wording varies by extraction model):

[DEC] Webhook timeout updated to 10 seconds (was 30)
[CON] Max fee 0.5%
[DEC] Use POST /v1/items
[CON] Always return JSON output.

No paid key? Use a free Groq tier or a local Ollama model:

# Groq (free tier available at console.groq.com)
llm = OpenAI(base_url="https://api.groq.com/openai/v1", api_key="gsk_...")

# Ollama (local, no key needed)
llm = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

# Together AI
llm = OpenAI(base_url="https://api.together.xyz/v1", api_key="...")

Why it's different

• Constraints and decisions are protected. [CON] and [DEC] patches are never dropped by dedup, fusion or pruning, and are selected first at render time.
• Revisions supersede old values. Change a value mid-chat and the current one survives, with the lineage baked in as (was X).
• Memory persists across chats. Save at the end of one chat, load at the start of the next, with no API cost for recall.
• Fixed budget, predictable cost. You set the token budget; the rendered context stays inside it.
• Universal CRR guarantee. CRR = 100% at any budget above the floor. Below the floor, raises BudgetTooSmallError instead of silently dropping criticals.

Universal CRR guarantee

from dspm.engine import DSPMEngine, BudgetTooSmallError

# Find the minimum safe budget for N critical patches
floor = DSPMEngine.min_budget_for_crr(22)   # → 59 tokens for 22 criticals

# Below the floor, raises a clear error instead of silently dropping
try:
    mem.set_budget(10)
    mem.get_context()
except BudgetTooSmallError as e:
    print(e)  # Budget 10 is below the absolute floor (59 tokens for 22 criticals)

5-stage compression ladder, which escalates only as needed:

Persistence: memory across chats and sessions

# Chat 1 — Monday
mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
mem.add_turn("user", "Building a budgeting app. Hard rules: must work offline, deadline Oct 20.")
mem.save("user_42_memory.json")

# Chat 2 — Thursday, new process
mem2 = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
mem2.load("user_42_memory.json")
mem2.get_context(query="What were my hard rules?")
# → [CON] must work offline
# → [CON] deadline Oct 20   ← recalled from Chat 1, zero API cost

mem2.add_turn("user", "The deadline moved to November 5.")
# → Oct 20 is superseded. November 5 replaces it.

Persistence safety: writes are atomic, and load(path, strict=True) raises CorruptNotebookError on corrupt files.

How it works

Raw conversation (648 tokens, 24 turns)
        ↓
Semantic extraction → 39 patches, 20 critical
        ↓
7-stage compression pipeline
        ↓
Compressed context (264 tokens @ budget 250) — 100% of criticals selected

Pipeline: fingerprint dedup → slot fusion → delta encoding → causal pruning → utility scoring → shadow selection → adaptive budget allocation.

Patch types

Regulatory and compliance requirements (PCI-DSS, HIPAA, GDPR, SOC 2, ISO 27001, FedRAMP) are classified as constraints by the extractor prompt.

Stats and metrics

print(mem.stats)

Testing

Run offline tests (no API key needed):

python tests/verify_crr.py
python tests/test_universal_budgets.py

Run live tests (API key required):

# Windows PowerShell
$env:GROQ_API_KEY = "gsk_..."

python tests/small_live_test_budgets.py   # 10-turn REST API conversation
python tests/large_live_test_budgets.py   # 24-turn fintech + 3 revisions

See tests/README.md for expected output and reproduction instructions.

API reference

DSPMMemory(budget, llm_client, model)

Methods

Properties

Advanced API

from dspm import DSPMEngine, extract_patches, merge_into_memory, SemanticPatch
from dspm.engine import BudgetTooSmallError

Repository layout

.
├── dspm/
│   ├── __init__.py      # public API
│   ├── patch.py         # SemanticPatch structure
│   ├── engine.py        # DSPMEngine v4.5 — 7-stage pipeline + CRR guarantee
│   ├── extractor.py     # LLM-based patch extraction
│   ├── client.py        # LLMClient + provider auto-detection
│   └── memory.py        # DSPMMemory high-level wrapper
├── tests/
│   ├── verify_crr.py              # quick CRR check
│   ├── test_universal_budgets.py  # offline sweep, 17 budgets
│   ├── small_live_test_budgets.py # 10-turn live test
│   ├── large_live_test_budgets.py # 24-turn live test with 3 revisions
│   └── README.md                  # reproduction instructions
├── pyproject.toml
├── README.md
└── LICENSE

Known Limitations

• Small budgets under-deliver readability. At budget 60, context is trimmed to bare numbers. CRR = 100% but human readability suffers. Recommended minimum for readable output: budget 100.
• Per-turn extraction cap. The extraction prompt allows at most 1 constraint and 1 decision per turn. If a single message states several hard rules, split them across turns.
• Revision matching is rule-based. Numeric, day-name and acronym-bridged revisions are handled. Other free-text revisions may coexist with stale values.
• Extraction depends on the LLM. Quality varies by model and domain. A constraint the extractor misses is not protected.
• No async support yet. add_turn() is synchronous. Planned for v0.3.0.
• Document QA. DSPM is designed for conversations, not document compression.
• Self-judged offline benchmark. Same model family for extraction and judging; not independently replicated. Run live tests with your own data using tests/large_live_test_budgets.py.

Paper

DSPM: A Critical-Retention Approach to Long-Context Memory Compression for LLM Conversations Dhruv Dubey, 2026 · Zenodo: 10.5281/zenodo.19438636

Recognition: Poster accepted at CAICS 2026 (Indo-US Workshop on AI & Cybersecurity), jointly organized by IISc Bengaluru and Argonne National Laboratory, funded by the U.S. Department of State.

Feedback

Found a case where a constraint or revision is lost? Open an issue with the conversation and the budget you used.

License

MIT © 2026 Dhruv Dubey