cachelint

cachelint is a Python package for investigating LLM prompt-cache misses. It compares request bodies offline and shows where the modeled prompt prefix changes.

Status: early alpha. Version 0.1.0a1 can ingest and compare raw request bodies through the Python API and CLI. SDK recording is planned for a later alpha release. See the release notes.

Requirements and current usage

Python 3.11 or newer is required. The initial CI matrix covers Python 3.11 through 3.14.

For development from this checkout:

python -m pip install --upgrade pip
python -m pip install --group dev -e .
cachelint --help
cachelint --version

The cachelint distribution, cachelint import package, and cachelint CLI share the same name. The core package has no runtime dependencies.

Analyze files from the CLI

Save one complete raw request body per nonblank line in a UTF-8 JSONL file, then run:

cachelint analyze requests.jsonl
cachelint analyze requests.jsonl --provider openai

To compare two separate JSON files, each containing one complete request body (pretty printing is allowed):

cachelint diff first.json second.json
cachelint diff first.json second.json --provider anthropic

The default provider choice is auto. Bodies without a provider-specific marker need --provider anthropic or --provider openai; conflicting markers are rejected. A comparison requires the same provider and exact model in every request. Output includes the provider, request count, prompt status, common prefix segment count, first safe divergence path and character offset when available, context categories, coverage warnings, and fixed findings. It never prints request values or excerpts. cachelint analyze compares every valid JSONL request as one group; one request reports insufficient_data.

Exit status 0 means a valid analysis, including a divergence or insufficient data. Status 2 means invalid arguments or input, including unreadable files; status 1 means an unexpected internal failure. Invalid input produces no partial report. A matching modeled prefix does not guarantee a provider cache hit. See the ingestion guide and comparison guide for input rules and finding meanings.

Request ingestion in Python

canonicalize_request validates one raw Anthropic Messages or OpenAI Chat Completions request body and returns ordered prompt segments, separate request context, and coverage warnings:

from cachelint import canonicalize_request

request = {
    "model": "demo",
    "system": "Follow the instructions.",
    "messages": [{"role": "user", "content": "Hello"}],
}
canonical = canonicalize_request(request)
print(canonical.provider)  # anthropic
print([segment.path for segment in canonical.segments])
# ['system', 'system', 'messages[0]']

Use provider="anthropic" or provider="openai" when a body has no provider-specific marker. InvalidRequestError reports invalid shapes without echoing request values. The segment value and context inventory do retain request content; protect them as you would the original body. See the ingestion guide for accepted JSONL input, validation rules, and limits.

Comparing request prefixes in Python

compare_requests accepts one or more canonical requests of the same provider and exact model. It returns the first difference in the modeled prompt prefix and reports context changes separately:

from cachelint import canonicalize_request, compare_requests

first = canonicalize_request(
    {
        "model": "demo",
        "system": "You are helpful.",
        "messages": [{"role": "user", "content": "Hello"}],
    }
)
second = canonicalize_request(
    {
        "model": "demo",
        "system": "You are careful.",
        "messages": [{"role": "user", "content": "Hello"}],
    }
)
result = compare_requests([first, second])
print(result.prompt_status)  # diverged
assert result.first_divergence is not None
print(result.first_divergence.path, result.first_divergence.offset)  # system 8

The result contains safe paths, difference kinds, character offsets where available, context categories, coverage warnings, and evidence-based findings. It contains no prompt excerpts or raw values. See the comparison guide for result semantics and privacy limits.

First analysis release scope

The 0.1.0a1 milestone targets offline analysis for Anthropic Messages and OpenAI Chat Completions request bodies. The accepted canonicalization contract and privacy handling guide the commands. A matching text prefix is evidence for diagnosis, not a guarantee of a provider cache hit.

The request comparison and privacy contract defines the first release's input and diagnostic behavior.

Request logs can contain prompts, credentials, personal data, and customer content. Do not commit raw logs or paste them into public issues.

Compatibility

Within the 0.x series, patch releases preserve documented CLI and Python API behavior. Minor releases may make incompatible changes with migration notes. Alpha APIs may evolve between prereleases; changes will be recorded in release notes. Only Python versions exercised in CI are claimed as supported.

See CONTRIBUTING.md for development and SECURITY.md for private vulnerability reporting.

Licensed under Apache-2.0. This project is independent of the Apache Software Foundation.