ecoai-python 0.1.3

Token-saving cache wrapper for OpenAI, Anthropic, and Google Gemini

EcoAI Python SDK

Token-saving cache wrapper for OpenAI, Anthropic, and Google Gemini.

Intercepts API calls, caches identical responses locally, and returns them instantly on repeat calls — saving tokens, cost, and CO₂.

Installation

pip install ecoai

# With provider extras
pip install "ecoai[openai]"
pip install "ecoai[anthropic]"
pip install "ecoai[gemini]"          # google-generativeai (GenerativeModel pattern)
pip install "ecoai[gemini-genai]"    # google-genai (genai.Client pattern)
pip install "ecoai[redis]"           # Redis storage backend
pip install "ecoai[all]"             # everything

Quick start

import os
from ecoai import EcoAI
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
eco = EcoAI(client=client, mode="dev")

# Use eco exactly like your regular OpenAI client
response = eco.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise the water cycle."}],
)
print(response.choices[0].message.content)

# Second call returns instantly from cache
response2 = eco.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise the water cycle."}],
)

Anthropic

from ecoai import EcoAI
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
eco = EcoAI(client=client, mode="dev")

response = eco.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain quantum entanglement."}],
)
print(response.content[0].text)

Google Gemini

EcoAI supports both the old google-generativeai SDK and the new google-genai SDK.

Old SDK (google-generativeai — GenerativeModel pattern):

import google.generativeai as genai
from ecoai import EcoAI

genai.configure(api_key=os.environ["GOOGLE_API_KEY"])
model = genai.GenerativeModel("gemini-1.5-flash")
eco = EcoAI(client=model)

response = eco.generate_content("What causes the northern lights?")
print(response.text)

New SDK (google-genai — genai.Client pattern):

from google import genai
from ecoai import EcoAI

client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])
eco = EcoAI(client=client)

response = eco.models.generate_content(
    model="gemini-2.0-flash",
    contents="What causes the northern lights?",
)
print(response.text)

Configuration

from ecoai import EcoAI, SemanticConfig

eco = EcoAI(
    client=client,
    mode="prod",                    # "dev" (no expiry) | "prod" (TTL-based)
    storage="sqlite",               # "sqlite" | "memory" | "redis"
    sqlite_path=".ecoai/cache.db",  # custom SQLite path
    ttl=3600,                       # seconds, prod mode only
    ttl_by_model={"gpt-4o": 7200},  # per-model TTL overrides
    log_usage=True,                 # write per-call records to usage.db
    caching_enabled=True,           # master on/off switch
    semantic=SemanticConfig(        # semantic (similarity-based) caching
        threshold=0.95,
        embedding_model="text-embedding-3-small",
    ),
)

Redis storage

eco = EcoAI(
    client=client,
    storage="redis",
    redis_url="redis://localhost:6379",  # or set ECOAI_REDIS_URL env var
)

Cache management

from ecoai import FlushFilter

eco.flush()                                    # clear all
eco.flush(FlushFilter(pattern="summarise*"))   # clear by prompt glob

Usage statistics

summary = eco.summary()
print(f"Hit rate: {summary.hit_rate:.0%}")
print(f"Tokens saved: {summary.tokens_saved:,}")
print(f"Cost saved: ${summary.cost_saved_usd:.4f}")
print(f"CO₂ saved: {summary.co2_saved_g:.2f}g")

records = eco.history(limit=20)
for r in records:
    print(r.timestamp, r.model, "HIT" if r.cache_hit else "MISS")

Semantic caching

Match semantically similar prompts even if worded differently:

eco = EcoAI(
    client=client,
    semantic=SemanticConfig(threshold=0.95),
)

# First call — cache miss
eco.chat.completions.create(model="gpt-4o", messages=[
    {"role": "user", "content": "What is the capital of France?"}
])

# Cache hit (cosine similarity ≥ 0.95)
eco.chat.completions.create(model="gpt-4o", messages=[
    {"role": "user", "content": "Tell me the capital city of France."}
])

Requires OpenAI API key for embeddings. Multimodal requests are always excluded.

Settings file

The EcoAI dashboard writes settings to .ecoai/settings.json. The Python SDK reads this file at startup as the lowest-priority config layer:

Priority: EcoAI(kwargs) > ECOAI_* env vars > .ecoai/settings.json > defaults

Production deployments

Deployment	Storage	Usage logging
Single Node server	sqlite ✅	enabled ✅
Multi-instance / load balanced	redis ✅	log_usage=False ⚠️
Serverless (Lambda, Cloud Functions)	redis ✅	log_usage=False ⚠️
Docker	sqlite ✅ (mount .ecoai/ as a volume)	enabled ✅

License

MIT