bedrockcache

Audit and fix Anthropic prompt caching when calling Claude on AWS Bedrock through any abstraction stack.

Why

Anthropic prompt caching cuts cost by ~90% on stable prefixes. On the native Anthropic API the SDK applies it automatically. But if you call Claude through Bedrock — directly via boto3, indirectly via LiteLLM, via AnthropicBedrock, as a Strands or pydantic-ai backend — caching breaks silently at one of the abstraction boundaries, and you find out via the bill.

A real Bedrock+LiteLLM stack ran up $37,901 in 9 days on uncached input tokens because each layer "nominally supported" caching but caching was not actually applying. bedrockcache catches that class of bug at request time.

Install

pip install bedrockcache

Audit a request before sending

from bedrockcache import audit, Backend

request = {
    "model": "bedrock/anthropic.claude-sonnet-4-5-v1:0",
    "messages": [{"role": "user", "content": "hi"}],
}
report = audit(request, Backend.LITELLM)
print(report.will_cache)        # False
print(report.errors)            # ['no `cache_control` on any message; LiteLLM has nothing to translate']
print(report.recommendations)   # ['add cache_control={...} either on the message dict or …']

Lock it in CI

def test_my_rag_caches():
    request = build_my_rag_request(query="...")
    audit(request, Backend.BEDROCK_CONVERSE).assert_caches()

assert_caches() raises AssertionError with the specific reason if any layer is dropping the directive.

CLI

$ bedrockcache audit request.json --backend litellm --strict
backend:           litellm
will_cache:        False
breakpoint_count:  0
reasons:
  [error] no `cache_control` on any message; LiteLLM has nothing to translate
recommendations:
  - add `cache_control={'type': 'ephemeral'}` either on the message dict or on the last content sub-item of stable prefixes.
$ echo $?
1

Supported backends

What it explicitly is not

• Not a router. Not a proxy. Not a generic LLM cost tracker.
• Bedrock-only by design.
• Not a replacement for Phoenix or Langfuse.
• Not coupled to any framework — zero runtime dependencies.

Sibling libraries

Three independent libraries that compose around Anthropic-on-Bedrock:

• bedrockcache (this) — static auditor for prompt caching.
• bedrockstack — Bedrock-aware retry policy, cost ledger, streaming-error normalization.
• ragvitals — 5-dim drift detection for the RAG pipelines these models live inside.

Roadmap

• v0.2: real Anthropic tokenizer in place of the char-based heuristic.
• v0.3: Reporter.from_cloudwatch(log_group=...) — sample CloudWatch InvokeModel logs and dollar-ize the cache miss rate.
• v0.4: suggest_breakpoints(system_prompt, model=...) — recommend optimal placement for long prefixes.

Develop

pip install -e ".[dev]"
pytest -v

License

MIT