langstate

langstate compresses older OpenAI-format conversation messages into a scaffold summary while preserving system messages and a configurable number of recent turns verbatim. Retention and size reduction depend on the conversation and summarizer.

validate() can record whether selected literal strings appear in the compressed output. It is deterministic and makes no model call, but it is a lexical check rather than a semantic-equivalence test.

What it does

from langstate import compress

compressed = compress(messages)
# Same OpenAI message shape; inspect retention before using the result
response = client.chat.completions.create(messages=compressed, model="gpt-4o")

The output is a valid OpenAI-format messages list:

• System prompts kept verbatim
• Last 4 turn-pairs kept verbatim
• Older turns compressed into a [SCAFFOLD STATE] system message via local or cloud model

Check selected literals — the receipt

validate returns a deterministic lexical receipt and makes no model call:

from langstate import compress, validate

compressed = compress(messages)

# Check the specific facts you care about:
receipt = validate(messages, compressed,
                   facts=["$4,000 budget", "launch May 5", "Dana"])

print(receipt.summary())   # counts literal strings found and approximate size change
print(receipt.dropped)     # selected strings not found in the output
assert receipt.ok          # optional gate on exact lexical retention

# Or heuristically select tokens (numbers, money, acronyms, names):
receipt = validate(messages, compressed)

A green receipt means only that the selected normalized strings occur in the output. It does not verify meaning, attribution, negation, completeness, or semantic equivalence. Auto-extraction is heuristic; pass an explicit facts= list for controlled checks. receipt.as_dict() gives a JSON-friendly record to log.

Install

python -m pip install 'langstate==0.2.0'

Requirements: Python 3.10+, no heavy dependencies (stdlib only). For local summarization, run Ollama locally:

ollama pull qwen3:4b

Adapters

langstate ships three built-in summarizer backends:

from langstate import compress
from langstate.adapters import build

# Local Ollama (default, no API key; uses local compute)
compressed = compress(messages)

# OpenAI
compressed = compress(messages, summarizer=build("openai"))

# Anthropic
compressed = compress(messages, summarizer=build("anthropic"))

# Any callable (prompt: str) -> str
compressed = compress(messages, summarizer=my_summarizer)

Configuration

compress(
    messages,
    preserve_recent=4,         # turn-pairs to keep verbatim (default: 4)
    min_turns_to_compress=6,   # skip compression for short conversations (default: 6)
    model="qwen3:4b",          # Ollama model when no summarizer is given
    summarizer=None,           # custom callable: (prompt: str) -> str
)

Choosing a model

The default is local qwen3:4b via Ollama. It requires a model download and local compute but no API key. Compression quality depends on the model and data; use explicit lexical checks plus an evaluation appropriate to the consequences of semantic errors.

Pick by what you're optimizing:

Switching is one argument:

compress(messages)                              # local qwen3:4b (default)
compress(messages, model="qwen3:14b")           # alternate local model; evaluate on your data
compress(messages, summarizer=build("openai"))  # optional provider adapter

No bundled result establishes a universal quality ranking. Evaluate summarizers on representative traffic, and pass model= explicitly when behavior stability matters.

Adapter probe

from langstate.adapters import probe, REGISTRY

for name in REGISTRY:
    print(probe(name))
# {"name": "local", "available": True, "latency_ms": 423, ...}
# {"name": "openai", "available": False, "reason": "OPENAI_API_KEY not set", ...}

License

Apache-2.0

About Hermes Labs

Hermes Labs builds AI audit infrastructure for teams deploying AI agents in regulated environments. All tools are released as open-source software — MIT or Apache-2.0, no SaaS tier. The audit work is paid; the code is not.

hermes-labs.ai

OSS audit stack