prompt-cache-optimizer

A zero-dependency Python micro-tool to structure LLM payloads for maximum prompt caching hits (OpenAI & Anthropic). Save up to 90% on API costs by ensuring your static contexts are perfectly formatted for caching.

The Problem

LLM providers like Anthropic and OpenAI now offer Prompt Caching, which drastically reduces costs for long prompts (like RAG documents or large system instructions). However, to trigger the cache, your payload must be strictly organized:

• Static content must be grouped perfectly at the front (Prefix Caching).
• Anthropic requires explicit cache_control breakpoints injected into specific blocks.

Manually managing this array structure in your code leads to messy boilerplate and missed cache hits.

The Solution

prompt-cache-optimizer is a pure standard library tool that takes your system prompt, your RAG documents, and your chat history, and automatically formats them into the perfect JSON payload required by either OpenAI or Anthropic to guarantee maximum cache utilization.

Installation

pip install prompt-cache-optimizer

Usage

from prompt_cache_optimizer import build_optimized_prompt
import anthropic

# Your heavy, static RAG context
docs = ["Long document 1...", "Long document 2..."]
history = [{"role": "user", "content": "What is in the docs?"}]

# Build the cache-optimized payload
payload = build_optimized_prompt(
    system_instruction="You are a helpful RAG assistant.",
    rag_documents=docs,
    chat_history=history,
    provider="anthropic" # or "openai"
)

# Unpack directly into the official SDK!
client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1024,
    **payload  # Automatically handles the `system` and `messages` arrays
)

Why zero-dependency?

If you just want to save API costs, you shouldn't have to install heavy frameworks like LangChain or LlamaIndex. This micro-tool operates in under 1ms, adds zero bloat to your deployment, and integrates seamlessly with the official openai and anthropic Python SDKs.