DSPM — memory for LLM conversations that keeps your constraints

Dynamic Semantic Patch Memory, by Dhruv Dubey · Python package: dspm-memory · Repo: Kernl

pip install dspm-memory

<<<<<<< HEAD Long chats drop the rule from turn 3 that everything else depends on. DSPM turns each turn into typed patches, protects constraints, decisions and revised values, and compresses the rest under a fixed token budget. Training-free. Works with any OpenAI-compatible endpoint.

Long chats drop the rule from turn 3 that everything else depends on. DSPM turns each turn into typed patches, protects constraints, decisions and revised values, and renders them under a fixed token budget. Training-free: no fine-tuning, just prompts and Python rules. Works with any OpenAI-compatible endpoint, sync or async.

fix-issue-2

Headline numbers

• 18-turn example conversation: 452 → 249 tokens, 100% of critical patches selected.
• 7 hand-authored 40-turn dialogues at a 250-token budget: about 83% fewer tokens, 100% critical retention (same model family extracted and judged; not independently replicated).
• Poster accepted at CAICS 2026 (Indo-US Workshop on AI & Cybersecurity), organized by IISc Bengaluru and Argonne National Laboratory.

Savings depend on conversation length: short conversations compress very little. Under very small budgets, some facts are stored but not rendered. See Known Limitations.

Who it's for: agents and chatbots whose long conversations must not lose a constraint, a decision, or a revised value (support bots, coding agents, assistants that remember a user's hard rules across sessions).

Who it's not for: summarizing or answering questions over documents. See Known Limitations.

<<<<<<< HEAD

Quickstart

=======

Quickstart (sync)

fix-issue-2

from openai import OpenAI
from dspm import DSPMMemory

llm = OpenAI(api_key="sk-...")   # any OpenAI-compatible endpoint
<<<<<<< HEAD
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")

memory.add_turn("user", "Hard rules: PCI-DSS compliant, max fee 0.5%, deadline Friday.")
# ... keep adding turns as the conversation goes ...
print(memory.get_context(query="What are the hard requirements?"))
print(memory.stats)   # token counts, CRR, TRR, LLM telemetry

DSPM uses an LLM to extract patches, so you need a client. No paid key? Use a free Groq tier or a local Ollama model:

# Groq (free tier available)
llm = OpenAI(base_url="https://api.groq.com/openai/v1", api_key="gsk_...")

# Ollama (local, no key needed)
llm = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

# Together AI
llm = OpenAI(base_url="https://api.together.xyz/v1", api_key="...")

Optional: semantic ranking

pip install "dspm-memory[semantic]"

Adds sentence-transformer query alignment for better ranking of non-critical patches (downloads PyTorch and a small embedding model on first use). Without it, DSPM uses a neutral alignment score.

Why it's different

• Constraints and decisions are protected. [CON] and [DEC] patches are never dropped by dedup, fusion or pruning, and are selected first at render time.
• Revisions supersede old values. Change a value mid-chat and only the current one survives, even across sessions.
• Memory persists across chats. Save at the end of one chat, load at the start of the next, with no API cost for recall.
• Fixed budget, predictable cost. You set the token budget; the rendered context stays inside it.

See a revision supersede an old value

A payment-API conversation where a value changes mid-chat. After compression only the current value remains and the hard rules are intact.

This short example uses budget=60 so the effect is visible, and it compresses only about 9% (61 → 56 tokens). The large savings show up on longer conversations, as in the benchmark below. Use 200–300 for real conversations.

memory = DSPMMemory(budget=60, llm_client=llm, model="gpt-4o-mini")
=======
mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")

mem.add_turn("user", "Always return JSON. Use POST /v1/items. Max fee 0.5%.")
mem.add_turn("assistant", "Webhook timeout set to 30 seconds.")
>>>>>>> fix-issue-2

# Later: the webhook timeout is revised
mem.add_turn("user", "Webhook timeout updated to 10 seconds.")

print(mem.get_context(query="What must the contract remember?"))
print(mem.stats)

<<<<<<< HEAD Example output (recorded with v0.1.17; exact wording varies by extraction model):

Illustrative output (exact wording varies by extraction model):

fix-issue-2

[DEC] Webhook timeout updated to 10 seconds (was 30)
[CON] Max fee 0.5%
[DEC] Use POST /v1/items
[CON] Always return JSON output.

No paid key? Use a free Groq tier or a local Ollama model:

# Groq (free tier available)
llm = OpenAI(base_url="https://api.groq.com/openai/v1", api_key="gsk_...")

# Ollama (local, no key needed)
llm = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

# Together AI
llm = OpenAI(base_url="https://api.together.xyz/v1", api_key="...")

DSPM uses an LLM to extract patches, so some client is required.

Quickstart (async, for FastAPI / aiohttp servers)

import asyncio
from openai import AsyncOpenAI
from dspm import DSPMMemory

async def main():
    client = AsyncOpenAI()
    mem = DSPMMemory(budget=250, llm_client=client, model="gpt-4o-mini")

    # Non-blocking: the event loop is free during the LLM call
    await mem.async_add_turn("user", "PCI-DSS compliant, deadline Friday.")
    print(mem.get_context())

asyncio.run(main())

async_add_turn() detects your client's async support:

• AsyncOpenAI: awaits create() directly.
• Older OpenAI SDKs: awaits acreate().
• Sync client: wraps the call in asyncio.to_thread(), so the event loop is still freed.

Optional: semantic ranking

pip install "dspm-memory[semantic]"

Adds sentence-transformer query alignment for better ranking of non-critical patches (downloads PyTorch and a small embedding model on first use). Without it, DSPM uses a neutral alignment score.

Requirements: Python 3.9+, tiktoken>=0.5.0, portalocker>=2.7 (installed automatically), and an OpenAI-compatible client.

Why it's different

• Constraints and decisions are protected. [CON] and [DEC] patches are never dropped by dedup, fusion or pruning, and are selected first at render time.
• Revisions supersede old values. Change a value mid-chat and the current one survives, with the lineage baked in as (was X).
• Memory persists across chats. Save at the end of one chat, load at the start of the next, with no API cost for recall.
• Fixed budget, predictable cost. You set the token budget; the rendered context stays inside it.

See a revision supersede an old value

mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")

mem.add_turn("user", "Cache TTL must be 60 seconds.")
mem.add_turn("user", "Actually make the TTL 300 seconds.")
mem.add_turn("user", "Change the TTL again, 120 seconds this time.")

print(mem.get_context(query="What is the TTL?"))
# e.g. [DEC] TTL is now 120 seconds (was 300 then 60)

The supersession rules recognize:

• Numeric revisions (60 → 300 → 120) with lineage chaining.
• Day-name revisions (Friday → Monday), with capitalization preserved.
• Acronym-bridged revisions (TTL ↔ Time-to-live (TTL)).
• Cross-type restatements (a user [CON] and an assistant [DEC] echo collapse into one).
• Bound-aware compatibility (at most 5 is not at least 5).

You can also test the rules offline, with no LLM call:

from dspm import DSPMMemory
from dspm.patch import SemanticPatch

mem = DSPMMemory(budget=250)
mem._merge_patch(SemanticPatch('p0', 0, 'decision', 'Webhook timeout set to 30 seconds', []))
mem._merge_patch(SemanticPatch('p1', 1, 'decision', 'webhook timeout now 10 seconds', []))
print(mem.critical_patches[0].payload)
# 'webhook timeout now 10 seconds (was 30)'

Try to break it: give it a conversation where a value changes several times under a very small budget, and open an issue if a constraint or revision is lost.

Persistence: memory across chats and sessions

Save the notebook when a chat ends, load it when the next one starts. Chat 2 remembers Chat 1, and revisions supersede old values across sessions:

# Chat 1 — Monday
<<<<<<< HEAD
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
memory.add_turn("user", "Building a budgeting app. Hard rules: must work offline, deadline Oct 20.")
# ... chat ...
memory.save("user_dhruv.json")

# Chat 2 — Thursday, new process, fresh start
memory = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini")
memory.load("user_dhruv.json")
memory.get_context(query="What were my hard rules?")
# → [CON] must work offline
# → [CON] deadline Oct 20        ← recalled from Chat 1, zero API cost

# Revisions work across chats too:
memory.add_turn("user", "The deadline moved to November 5.")
# → Oct 20 is superseded. November 5 replaces it.

Save files are portable JSON, written atomically (a crash mid-save can't corrupt the notebook), and merge-on-load means saved revisions supersede stale values. One file per user = each person's long-term memory.

mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini") mem.add_turn("user", "Building a budgeting app. Hard rules: must work offline, deadline Oct 20.") mem.save("user_42_memory.json")

Chat 2 — Thursday, new process

mem2 = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini") mem2.load("user_42_memory.json") # merge semantics mem2.get_context(query="What were my hard rules?")

→ [CON] must work offline

→ [CON] deadline Oct 20 ← recalled from Chat 1, zero API cost

mem2.add_turn("user", "The deadline moved to November 5.")

→ Oct 20 is superseded. November 5 replaces it.

**Safety**

- `save()` and `load()` take an exclusive file lock (via `portalocker`), so two processes can't race on the same file.
- Writes are atomic, and the previous file is backed up to `<path>.bak` before overwriting.
- If `load()` finds a corrupt file, it is quarantined to `<path>.corrupt.<timestamp>` instead of being overwritten.
- `load(path, strict=True)` raises `CorruptNotebookError` instead of skipping silently:

```python
from dspm.persistence import CorruptNotebookError

try:
    mem2.load("user_42_memory.json", strict=True)
except CorruptNotebookError as e:
    print(f"Notebook corrupt, quarantined: {e}")

fix-issue-2

Saved: the patches, turn count and token counters. Not saved: the supersession/dedup journals from dump_store(include_history=True), the last rendered context, and LLM telemetry. These start empty after load().

How it works

<<<<<<< HEAD DSPM converts each conversation turn into typed semantic patches (constraint, decision, code, equation, entity, structure) and compresses them under a fixed token budget. Critical patches (constraints and decisions) are structurally protected in the store and are selected first when the context is rendered.

DSPM converts each conversation turn into typed semantic patches and compresses them under a fixed token budget. Critical patches (constraints and decisions) are structurally protected in the store and are selected first when the context is rendered.

fix-issue-2

Raw conversation (452 tokens, 18 turns)
        ↓
Semantic extraction → 28 patches, 18 critical
        ↓
6-stage compression pipeline
        ↓
Compressed context (249 tokens) — 100% of criticals selected

The 6-stage pipeline: dedup → slot fusion → delta encoding → causal pruning → utility scoring → shadow selection. (A 7th stage, adaptive per-type budgeting, is designed but not wired in this release.)

<<<<<<< HEAD

Patch types

Regulatory, compliance and security requirements (PCI-DSS, HIPAA, GDPR, SOC 2, ISO 27001, FedRAMP) are classified by the extractor prompt as constraints, not entities. Classification depends on the extraction model.

Stats and metrics

print(mem.stats)

trr vs trr_by_conversation. A 55-token conversation might extract to 16 patch tokens and deliver 16 context tokens:

trr can be zero or negative on short conversations, because patches plus rendering overhead can outweigh the raw text. Use trr_by_conversation when you want to know what you saved.

fix-issue-2

The guarantee

What DSPM is built to guarantee, and where it stops:

• In the store: [CON] and [DEC] patches are never dropped by deduplication, fusion, or pruning.
• At render time: criticals are selected first. Under budget pressure their payloads are trimmed numbers-first, with units, negations and comparators protected where the budget allows.
• Revisions: when a constraint is revised mid-conversation, the new value supersedes the old, and the replaced values are kept as (was X). Under a starvation budget the revision story survives in minimal form where the budget allows, e.g. 10 (was 30).
• Last resort: a critical is dropped from the rendered context only when every critical is already at its word floor and the budget still cannot hold them. It stays in the store.
• Extraction failures: transient API failures (rate limits, 5xx) and empty responses are retried with backoff (three attempts). Use strict_llm=True and memory.stats to see failures.
• Ablation result: removing the shadow-selection mechanism collapses CRR from 100% to 37.9%, isolating the retention behavior to a single identifiable component.

Important limits of this guarantee:

• It covers what the extraction model captures as a constraint or decision. If the extractor misses or misclassifies something, DSPM cannot protect it.
• CRR is the share of extracted critical patches selected into the context. It is not the share of rules the user stated, and not the share of facts a reader can recover from the text. A rule the extractor never emitted does not count against CRR.
• At small budgets a selected line can be trimmed so far that it loses its unit, operator or subject (for example must not exceed 90 or 120 (was 300)).

Results

Benchmark suite (v0.1.17)

Tested across 7 domains × 40 turns each:

CRR = Critical Retention Rate (extracted critical patches selected into the context). TRR = Token Reduction Ratio.

Provenance: measured on 7 hand-authored 40-turn technical dialogues (API design, ML ops, IoT, security IR, supply chain, clinical workflow, project management); extraction and judging by GLM via OpenRouter. The same model family performed both extraction and judging, and the benchmark has not been independently replicated. It was run on v0.1.17, not on the latest release. Numbers vary by extraction model and domain; check your own with memory.stats.

Diagnostic runs (v0.1.18, small sample)

One synthetic 48-message alerting-pipeline conversation (12 core messages, 36 filler messages, one revision, one bound), openai/gpt-oss-20b on Groq, two runs per setting. "Facts delivered" counts how many of 8 ground-truth facts a fixed substring scorer finds in the context rendered after message 48.

What we observed:

• The memory store held all core facts in almost every run; the loss was mostly at render time under a small budget.
• With feedback on, results varied between identical runs (7/8 vs 3/8 at budget 120); with it off they did not.
• echo_guard blocked the fact-deleting case, but it did not fix budget crowding: filler decisions are also extracted as critical and compete for slots.

This is one conversation, a few runs per setting, one model, and a substring scorer. Treat it as a diagnosis, not a benchmark.

Compare it yourself

experiments/benchmark_compare.py measures DSPM against a same-budget rolling summary on fixed scenarios with shared gold checks. The offline mock (--mode mock) prints ground-truth retention per scenario; --mode live runs a real extractor, a true rolling summary, and an LLM judge over a dialogues file you supply (--dialogues), reporting mean ± std across dialogues.

python experiments/benchmark_compare.py --mode mock
python experiments/benchmark_compare.py --mode live --base-url https://api.groq.com/openai/v1 \
    --model openai/gpt-oss-20b --dialogues dialogues.json

Mock-mode reference: DSPM 100% mean retention vs rolling-window 70% at budget 250. These are pipeline-check numbers from a canned extractor and a string-match judge, not extraction quality; run --mode live for that.

experiments/bench_kv_cache.py answers whether a stable patch context keeps more of the prompt prefix byte-stable than a same-budget raw last-K window (reference: 85.5% vs 70.7% over 40 turns at budget 250). experiments/bench_kv_live.py measures cached-token reuse on a real endpoint. Neither harness yet accounts for extraction latency. See experiments/README.md.

<<<<<<< HEAD

python experiments/bench_kv_cache.py
python experiments/bench_kv_cache.py --sweep
python experiments/bench_kv_live.py

Diagnostic scripts (v0.1.18): scripts/diag_48.py runs the 48-message diagnostic at budgets 120 and 60 and saves the rendered context, store dump and stats at messages 12/24/36/48; scripts/analyze_diag.py classifies each ground-truth fact as in the store, in the context, both, or neither. Both need GROQ_API_KEY. Flags: --no-feedback, --echo-guard, --fence, --out-dir.

=======

fix-issue-2

Known Limitations

<<<<<<< HEAD

• Small budgets under-deliver rendered facts. At budget 60 in our diagnostic conversation, only 2–3 of 8 facts appeared in the rendered context even though the store held them. The renderer trims selected lines to a word floor, so units, operators or subjects can be lost (for example must not exceed 90 or 90 days instead of must not exceed 90 days).
• Filler can crowd out core facts. Routine status messages (such as "pod limit raised to 6Gi") can be extracted as decisions, which are critical, and then compete with older hard rules for slots. Selection has no notion of importance beyond recency and alignment.
• Duplicate restatements: when the same fact is stated as both a constraint and a decision, the pair is not merged and both consume budget (observed as [CON] within 2 plus [DEC] 2-second delivery).
• Non-numeric facts can be missed: identity facts without numbers (for example the name of a plan) were extracted in some runs and not others, and were never rendered.
• Feedback context (feedback_context=True) can cause the extractor to re-extract trimmed context lines; see CHANGELOG.md. echo_guard reduces the damage but is experimental: its thresholds were tuned on a single failure, and it can drop a genuine update that shares no wording with its turn.
• Persistence: load() restores patches but not the history journals, the last rendered context, or telemetry.
• Numberless obligation echoes: when a requirement is restated without any numeral (e.g. "all payments must be PCI-DSS compliant" restated as "PCI-DSS compliance"), no collapse rule can pin the pair, because the restatement rules are number-anchored. Structured slot identity is planned for a later release. =======
• Small budgets under-deliver rendered facts. At budget 60 in our diagnostic conversation, only 2–3 of 8 facts appeared in the rendered context even though the store held them. The renderer trims selected lines to a word floor, so units, operators or subjects can be lost.
• Per-turn extraction cap. The standard extraction prompt allows at most one constraint and one decision per turn (two each for fact-dense turns). If a single message states several hard rules, the extras may not be stored. Split rules across turns, or test dense=True on your data.
• Filler can crowd out core facts. Routine status messages can be extracted as decisions, which are critical, and then compete with older hard rules for slots.
• Revision matching is rule-based. Numeric, day-name and acronym-bridged revisions are handled; other numberless values (names, plan tiers, free-text settings) can coexist with a stale value. The restatement rules are number-anchored, and structured slot identity is planned for a later release.
• Duplicate restatements. The same fact stated twice with different wording (for example "Deadline updated to Monday" and "Deadline moved to Monday") can both be rendered and consume budget.
• Extraction depends on the LLM. Quality varies by model and domain, a constraint the extractor misses is not protected, and extraction is not fully deterministic even at temperature 0. In our runs openai/gpt-oss-20b on Groq intermittently returned empty extraction output (3 of 14 calls in one run); gpt-oss-120b and qwen3.8-27b did not in the same test. Use strict_llm=True to surface empties.
• Feedback context (feedback_context=True) can cause the extractor to re-extract trimmed context lines. echo_guard reduces the damage but is experimental.

fix-issue-2

• Polarity under starvation: negations and comparators are protected down to the 2-word floor, but at the floor a comparator phrase can degrade to its bare value.
• Persistence: load() restores patches but not the history journals, the last rendered context, or telemetry.
• Document QA: DSPM is designed for conversations. In head-to-head document QA tests it lost to a query-aware compressor. Use it for dialogue memory, not for compressing papers or contracts.
• Benchmark scope: 7 hand-authored dialogues plus one synthetic diagnostic conversation, self-judged, run on v0.1.17. Treat the numbers as a baseline, not a third-party evaluation.
• Still early: version 0.1.x, Beta. APIs may change.

Document mode: dense extraction

# automatic — fact-dense turns route through the dense funnel:
mem.add_turn("user", "p95 latency 200ms, fee 0.5%, timeout 30 seconds, budget 250 tokens")

# documents:
n_chunks = mem.add_document(open("paper.txt", encoding="utf-8").read())

# force dense mode:
mem = DSPMMemory(budget=250, llm_client=llm, model="gpt-4o-mini", dense=True)

API reference

DSPMMemory(budget, llm_client, model, ...)

The budget can be changed at any time:

mem.budget = 100
mem.set_budget(100)   # explicit alternative — same effect

Methods

Properties

Repository layout

.
<<<<<<< HEAD
├── src/dspm/
│   ├── config.py
│   ├── patch.py
│   ├── extractor.py
│   ├── memory.py
│   ├── engine.py
│   └── persistence.py
├── tests/                        # 151 offline tests (pytest)
├── scripts/                      # live smoke gates and diagnostics (need GROQ_API_KEY)
├── experiments/
│   ├── benchmark_compare.py
│   ├── bench_kv_cache.py
│   ├── bench_kv_live.py
│   └── results/
├── examples/
│   └── basic_usage.py
=======
├── src/dspm/            # config, patch, extractor, memory, engine, persistence
├── tests/               # offline tests (pytest)
├── scripts/             # live smoke gates and diagnostics (need an API key)
├── experiments/         # benchmark_compare.py, bench_kv_cache.py, bench_kv_live.py
├── examples/
>>>>>>> fix-issue-2
├── pyproject.toml
├── CHANGELOG.md
├── LICENSE
└── README.md

<<<<<<< HEAD Run the offline tests with pytest tests -q. Do not run a bare pytest from the repo root: some scripts under scripts/ call a live API when collected.

Run the offline tests with python -m pytest tests -q. Do not run a bare pytest from the repo root: some scripts under scripts/ call a live API when collected.

fix-issue-2

Paper

DSPM: A Critical-Retention Approach to Long-Context Memory Compression for LLM Conversations Dhruv Dubey, 2026 Zenodo: 10.5281/zenodo.19438636

Recognition: Poster accepted at CAICS 2026 (Indo-US Workshop on AI & Cybersecurity), jointly organized by IISc Bengaluru and Argonne National Laboratory, funded by the U.S. Department of State.

Feedback

Found a case where a constraint or revision is lost? Please open an issue with the conversation and the budget you used.

License

<<<<<<< HEAD MIT © 2026 Dhruv Dubey

MIT © 2026 Dhruv Dubey

fix-issue-2