chatcram

Keep a long chat history within a token budget — by summarizing the old middle and keeping the system prompt + recent turns verbatim. Tiny, zero-dependency, framework-agnostic. Bring your own summarizer.

As a conversation grows, you eventually blow past the context window. Dropping old turns loses information; keeping everything is impossible. chatcram collapses the older middle into a single summary while preserving what matters most — the system prompt and the most recent turns.

from chatcram import Compactor

# `summarize` is any callable you provide — usually an LLM call
compactor = Compactor(budget=4000, summarize=my_llm_summarizer, keep_recent=1500)

result = compactor.compact(messages)   # list of {"role", "content"} dicts

for m in result.messages:
    print(m["role"], "->", m["content"][:60])

print(result.summarized)    # True if the middle was collapsed
print(result.used_tokens)   # tokens in the compacted history

What you get back:

• System messages — always kept, verbatim, at the front.
• A single summary message — the older middle, collapsed via your summarizer.
• Recent turns — the latest keep_recent tokens, kept verbatim.

Why

• Zero dependencies. Pure Python. A fast characters-per-token heuristic by default; plug in tiktoken or any tokenizer for exact counts.
• Bring your own summarizer. Any str -> str callable (an LLM call, a local model, anything). No provider lock-in, no hidden API calls.
• Framework-agnostic. Works on plain message dicts — not tied to LangChain or LlamaIndex.
• Composes with contextcram. Compact the history, then pack it into a full prompt budget.

Installation

pip install chatcram
# optional: exact token counts via tiktoken
pip install "chatcram[tiktoken]"

How it works

from chatcram import Compactor

def summarize(transcript: str) -> str:
    # call your LLM here; return a short summary string
    return my_client.complete(f"Summarize this conversation:\n{transcript}")

compactor = Compactor(
    budget=4000,          # if the history exceeds this, compact it
    summarize=summarize,
    keep_recent=1500,     # tokens of the most recent turns to keep verbatim
)

result = compactor.compact(messages)
messages = result.messages   # ready to send to the model

If the history is already under budget, it's returned unchanged (summarized=False). The most recent turn is always kept, even if it alone exceeds keep_recent.

Pairs with contextcram

from chatcram import Compactor
from contextcram import Packer

history = Compactor(budget=3000, summarize=summarize).compact(messages).messages

ctx = (
    Packer(model="gpt-4o", reserve=1500)
    .add(SYSTEM_PROMPT, priority="required")
    .add([f"{m['role']}: {m['content']}" for m in history], priority="high", strategy="trim")
    .add(retrieved_docs, priority="medium", strategy="drop")
    .fit()
)

Alternatives

Summarizing old turns isn't new, but it's almost always bundled into a framework or a heavyweight memory platform. chatcram is the standalone, dependency-free building block:

Choose chatcram when you want a tiny, framework-agnostic helper that summarizes the old middle of a conversation, with your own summarizer and no dependencies.

Development

git clone https://github.com/Waelr1985/chatcram.git
cd chatcram
uv sync
uv run pytest
uv run ruff check .
uv run mypy

License

MIT