composecache 0.1.3

Compositional semantic caching for LLM APIs and RAG pipelines

ComposeCache Python Package

Python package for compositional semantic caching over LLM requests.

Published package name on PyPI: composecache.

Quick Start

1. Prerequisites

• Python 3.10+
• PostgreSQL with pgvector enabled
• OpenAI API key

2. Install

python -m venv .venv
source .venv/bin/activate
pip install composecache

3. Configure Environment

export DATABASE_URL="postgresql://dev:dev@localhost:5432/composecache"
export OPENAI_API_KEY="sk-..."

ComposeCache automatically connects and runs its schema migration on first use.

4. Run

python - <<'PY'
from composecache import ComposeCache

cache = ComposeCache(
    database_url="postgresql://dev:dev@localhost:5432/composecache",  # or os.environ["DATABASE_URL"]
    openai_api_key="YOUR_OPENAI_KEY",  # or os.environ["OPENAI_API_KEY"]
)

request = {
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Compare GDP of France and Germany"}],
}

first = cache.complete(request)
second = cache.complete(request)

print("First call cache type:", first["cache_type"])
print("Second call cache type:", second["cache_type"])
print("Answer:", second["content"])
PY

5. Useful Response Fields

• content: model answer text
• cache_type: exact, semantic, partial, or miss
• tokens_saved: estimated tokens avoided via cache reuse
• cost_saved: estimated cost savings