AgentCache Python Client

Official Python client for AgentCache.ai - The Global Edge Cache for LLMs.

Installation

pip install agentcache

Usage

Standard Completion

import agentcache

# Drop-in replacement for OpenAI call logic
response = agentcache.completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello world"}],
    provider="openai"  # optional, defaults to openai
)

if response and response.get('hit'):
    print("Cache HIT:", response['response'])
else:
    print("Cache MISS - Call your LLM here")

Streaming

AgentCache supports streaming responses, making it compatible with chat UIs that expect Server-Sent Events (SSE).

stream = agentcache.completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Write a poem"}],
    stream=True
)

if stream:
    print("Cache HIT (Streaming):")
    for chunk in stream:
        content = chunk['choices'][0]['delta'].get('content', '')
        print(content, end="", flush=True)

Reasoning Cache (NEW in v0.3.0)

Cache reasoning traces for o1, Kimi, and DeepSeek models:

response = agentcache.completion(
    model="o1-preview",
    messages=[{"role": "user", "content": "Analyze this contract..."}],
    strategy="reasoning_cache"
)

if response and response.get('cached'):
    print("Reasoning trace retrieved from cache")

Multimodal Cache (NEW in v0.3.0)

Cache 3D meshes, images, and audio for generative models:

response = agentcache.completion(
    model="sam-3d-body",
    messages=[{
        "role": "user",
        "content": "Generate 3D model",
        "file_path": "input.jpg"
    }],
    strategy="multimodal"
)

if response and response.get('cached'):
    asset_data = response['asset']
    print(f"Retrieved cached 3D asset: {len(asset_data['vertices'])} vertices")

Environment Variables

Set AGENTCACHE_API_KEY in your environment, or pass api_key to the constructor.

from agentcache import AgentCache

client = AgentCache(api_key="ac_live_...")