llm-cache-lite

A lightweight, decorator-based caching library for LLM (Large Language Model) API calls. Save money, reduce API latency, and speed up your AI development cycle instantly.

Why use llm-cache-lite?

When developing AI applications, you often send the same prompts over and over again. This wastes API credits and slows down your testing. llm-cache-lite solves this by intercepting your AI functions and caching the responses in a local SQLite database.

• Universal: Works seamlessly with OpenAI, Anthropic, Gemini, local models, or any custom function.
• Async Ready: Native support for async/await functions
• Zero Boilerplate: Just add the @llm_cache decorator. No complex clients to initialize.
• Smart TTL: Automatically expire and invalidate old cached responses.
• Developer Friendly: Type-hinted and strictly validated with Pydantic.

Installation

pip install llm-cache-lite

Quick Start

Standard (Sync) Usage

import time
from llm_cache_lite import llm_cache, CacheSettings

# Optional: Configure database path and TTL (Time-To-Live in seconds)
# Here, the cache will expire after 1 hour (3600 seconds)
settings = CacheSettings(db_path="my_ai_cache.db", ttl_seconds=3600)

@llm_cache(settings=settings)
def ask_ai(prompt: str) -> str:
    # Simulate an expensive API call to OpenAI/Claude/Gemini etc.
    print(f"Calling real AI for: {prompt}")
    time.sleep(2) 
    return f"AI response to: {prompt}"

# First call: Executes the function (takes 35 seconds)
print(ask_ai("What is Python?"))

# Second call: Returns instantly from the local SQLite cache! (~0.001 seconds)
print(ask_ai("What is Python"))

Asynchronous (Async) Usage

import asyncio
from llm_cache_lite import llm_cache, CacheSettings

# Optional: Configure database path and TTL (Time-To-Live in seconds)
# Here, the cache will expire after 1 hour (3600 seconds)
settings = CacheSettings(db_path="my_ai_cache.db", ttl_seconds=3600)

@llm_cache(settings=settings)
async def async_ask_ai(prompt: str) -> str:
    # Simulate an expensive API call to OpenAI/Claude/Gemini etc.
    print(f"Calling real AI for: {prompt}")
    await asyncio.sleep(2)
    return f"Async AI response to: {prompt}"

async def main():
    # First call: Executes the function (takes 35 seconds)
    print(await async_ask_ai("What is the speed of sound?"))
    
    # Second call: Returns instantly from the local SQLite cache! (~0.001 seconds)
    print(await async_ask_ai("What is the speed of sound?"))
    
asyncio.run(main())

Configuration

The CacheSettings object uses Pydantic to ensure your configuration is always valid. It accepts the following parameters:

License

This project is licensed under the MIT License.