cache-doctor

The checkup your semantic cache never had.

LLM gateways all ship semantic caches now — but every tool on the market only operates the cache (store, lookup, evict). Nobody diagnoses it. cache-doctor is the first CLI that performs a health check on a semantic cache: it reads your cache event log and reports hit rate, expiry waste, key fragmentation, and — the silent killer — cache pollution, where a lightly rewritten prompt matches a cached response it should never have matched.

Differentiation

Think of it as the relationship between a database and a database health monitor: the cache is the patient, cache-doctor is the checkup instrument.

Install

pip install cache-doctor

Zero dependencies, stdlib only. Python ≥ 3.9. Works fully offline — your prompts never leave the machine.

Usage

Point it at a JSONL cache log — one event per line:

{"ts": "2026-10-11T01:00:01Z", "prompt": "What is the capital of France?",
 "cache_key": "k001", "hit": false, "miss_reason": "no_key",
 "threshold": 0.9, "ttl_seconds": 3600}
{"ts": "2026-10-11T01:00:22Z", "prompt": "Capital of France?",
 "cache_key": "k001", "hit": true, "similarity": 0.91,
 "threshold": 0.9, "ttl_seconds": 3600, "cache_age_seconds": 60}

# Full health report + suggestions
cache-doctor analyze cache-log.jsonl

# Machine-readable metrics
cache-doctor analyze cache-log.jsonl --json

# Suggestions only (from a log, or from saved --state)
cache-doctor suggest --log cache-log.jsonl
cache-doctor analyze cache-log.jsonl --state state.json
cache-doctor suggest --state state.json

What it reports

• Hit rate — overall plus a rolling hourly trend, so you can see when it degraded.
• Miss reasons — no_key / expired / low_similarity, with expired-miss share.
• Pollution signals — borderline hits (similarity within +0.05 of threshold), ground-truth wrong hits (flag_wrong: true), and near misses.
• Key fragmentation — the same normalized prompt stored under multiple keys (a prompt-normalization bug wasting storage and diluting hits).
• Stale serving — hits served from entries past 90% of their TTL.
• Suggestions — severity-ranked fixes (POLLUTION_CONFIRMED, THRESHOLD_STRICT, EXPIRY_WASTE, KEY_FRAGMENTATION, STALE_SERVING, …).

Log schema

Example

$ cache-doctor analyze examples/sample-log.jsonl

cache-doctor  health report
============================================
events analyzed :         16
cache hits      :          6     37.5%  [###########-------------------]
cache misses    :         10
similarity thr. :        0.9

miss reasons:
  expired             2    20.0%
  low_similarity      3    30.0%
  no_key              5    50.0%

pollution signals:
  borderline hits :      3  ( 50.0% of hits)
  wrong hits      :      1   <- flag_wrong=true
  near misses     :      3  ( 30.0% of misses)
...

Development

python3 -m unittest discover -s tests -p 'test_*.py'

License

MIT — see LICENSE.