django-llm-cache ⚡
A production-grade, modular, drop-in caching layer for LLM prompts and completions in Django applications.
Stop burning thousands of dollars every month on duplicate LLM API tokens across OpenAI, Gemini, Groq, and Anthropic.
💡 The Problem: The Duplicate Token Burn
Modern AI-enabled web apps, SaaS platforms, and enterprise dashboards suffer from a hidden margin killer: repetitive LLM prompts.
- Users repeatedly ask the same questions in chatbots and customer support portals.
- Dashboard widgets and analytics summary cards refresh on every page reload.
- Automated workflow triggers re-process identical document fragments.
- Rate limits on fast inference engines (like Groq) are hit unnecessarily by repeated queries.
💰 Return on Investment (ROI) Breakdown
Here is what caching 35% of identical queries looks like for a startup serving 100,000 queries per day:
| Metric | Without Cache | With django-llm-cache |
Savings / Advantage |
|---|---|---|---|
| API Calls to LLM Provider | 100,000 / day | 65,000 / day | 35,000 fewer API calls daily |
| Average Response Latency | ~1,200 ms | 3 ms (Cached Hits) | 99.7% latency drop on hits |
| Monthly Token Cost (GPT-4o) | ~$15,000 / mo | ~$9,750 / mo | $5,250 / month saved ($63,000 / yr) |
| Rate Limit Headroom | High risk of 429 errors | Unlocked 35%+ extra throughput | Zero throttle downtime |
| Database/Cache Failures | Typically crashes app | Fails open to live API | Zero downtime guarantee |
🏗️ Architecture & Request Lifecycle
sequenceDiagram
autonumber
actor Client as User / Frontend
participant View as Django View / DRF
participant Cache as django-llm-cache (LLMResponseCache)
participant Redis as Django Cache Backend (Redis / Memcached)
participant LLM as LLM Provider (Groq / OpenAI / Gemini)
Client->>View: POST /api/chat/ {prompt: "Summarize Q3 report"}
View->>Cache: cache.get(prompt, provider="groq", model="llama-3.3-70b-versatile")
Cache->>Cache: Generate SHA-256 Key(provider + model + cleaned prompt)
Cache->>Redis: GET key
alt Cache Hit (Instant Return)
Redis-->>Cache: Return cached response payload
Cache-->>View: Cached response
View-->>Client: 200 OK (Latency: ~3ms, Cost: $0.00)
else Cache Miss / First Query
Redis-->>Cache: None
Cache-->>View: None
View->>LLM: Live API call (Inference)
LLM-->>View: Return LLM response
View->>Cache: cache.set(prompt, provider, model, response, timeout=300)
Cache->>Redis: SET key response (TTL 300s)
View-->>Client: 200 OK (Latency: ~1200ms)
else Cache Backend Offline / Network Error
Redis--xCache: ConnectionError / Timeout
Note over Cache: Caught exception, logged warning.<br/>Fails open without crashing!
Cache-->>View: None
View->>LLM: Fallback directly to live API
LLM-->>View: Return response
View-->>Client: 200 OK (Zero interruption to end user)
end
✨ Key Features
- Deterministic SHA-256 Hashing: Strips prompt whitespace and creates canonical keys combining
provider,model_name, prompt content, and optional deterministic hyperparameters (temperature,system_prompt). - Resilient Fail-Open Design: If your Redis server crashes, your database runs out of connections, or network lag occurs,
django-llm-cachecatches the error, logs it via Python's standardlogging, and cleanly returnsNone. Your live user request never crashes. - Lazy Django Imports: Safe to import in CLI scripts, background workers, or standalone scripts without throwing
AppRegistryNotReadyorImproperlyConfigured. Emits clean, non-fatal warnings when settings are unconfigured. - Pluggable Cache Backends: Works with any Django cache backend (
django-redis, Memcached, Local Memory, FileSystem, or Database cache). You can specify dedicated cache partitions using thecache_aliasparameter. - Flexible Expiration: Configurable default TTL (default: 300 seconds) with per-call overrides.
📦 Installation
pip install django-llm-cache
Ensure Django (>=3.2) is configured in your project settings:
# settings.py
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.redis.RedisCache",
"LOCATION": "redis://127.0.0.1:6379/1",
}
}
🚀 Quickstart & Code Examples
1. Integration with Groq Python SDK
from groq import Groq
from django_llm_cache import LLMResponseCache
# Initialize cache manager (defaults to 300s expiration)
llm_cache = LLMResponseCache(default_timeout=300)
client = Groq()
def ask_groq(prompt: str) -> str:
provider = "groq"
model = "llama-3.3-70b-versatile"
# 1. Check local cache
cached_content = llm_cache.get(prompt, provider=provider, model_name=model)
if cached_content is not None:
return cached_content
# 2. Cache miss: Execute live API call
chat_completion = client.chat.completions.create(
messages=[{"role": "user", "content": prompt}],
model=model,
)
result_text = chat_completion.choices[0].message.content
# 3. Store result in cache for future calls
llm_cache.set(prompt, provider=provider, model_name=model, response=result_text)
return result_text
2. Integration with OpenAI Python SDK
from openai import OpenAI
from django_llm_cache import LLMResponseCache
llm_cache = LLMResponseCache(default_timeout=600) # 10 minutes TTL
client = OpenAI()
def ask_openai(prompt: str) -> str:
provider = "openai"
model = "gpt-4o"
# One-line lookup or fetch pattern
def fetch_from_api():
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return response.choices[0].message.content
# Automatically checks cache; calls fetch_from_api() only on cache miss!
return llm_cache.get_or_set(
prompt=prompt,
provider=provider,
model_name=model,
default_callable=fetch_from_api,
)
3. Integration with Google Gemini SDK
from google import genai
from django_llm_cache import LLMResponseCache
llm_cache = LLMResponseCache(default_timeout=900)
client = genai.Client()
def ask_gemini(prompt: str) -> str:
provider = "gemini"
model = "gemini-2.5-flash"
cached_text = llm_cache.get(prompt, provider, model)
if cached_text:
return cached_text
response = client.models.generate_content(
model=model,
contents=prompt,
)
llm_cache.set(prompt, provider, model, response.text)
return response.text
4. Integration in a Django REST Framework (DRF) View
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from django_llm_cache import LLMResponseCache
from groq import Groq
cache = LLMResponseCache(default_timeout=600, key_prefix="customer_chat")
groq_client = Groq()
class AssistantView(APIView):
def post(self, request):
user_prompt = request.data.get("prompt", "").strip()
if not user_prompt:
return Response({"error": "Prompt is required"}, status=status.HTTP_400_BAD_REQUEST)
provider = "groq"
model = "llama-3.3-70b-versatile"
# Check cache
cached_response = cache.get(user_prompt, provider, model)
if cached_response is not None:
return Response({"answer": cached_response, "source": "cache"})
# Call live LLM
completion = groq_client.chat.completions.create(
messages=[{"role": "user", "content": user_prompt}],
model=model,
)
answer = completion.choices[0].message.content
# Cache response
cache.set(user_prompt, provider, model, answer)
return Response({"answer": answer, "source": "live_api"})
⚙️ Advanced Configuration
Dedicated Redis Cache Alias
If you want to isolate your LLM cache to a specific Redis database or Redis cluster without polluting your session or template cache:
# settings.py
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
},
"llm_redis": {
"BACKEND": "django.core.cache.backends.redis.RedisCache",
"LOCATION": "redis://127.0.0.1:6379/2",
"TIMEOUT": None, # Handled per key by django-llm-cache
},
}
Then pass the alias to LLMResponseCache:
cache = LLMResponseCache(
default_timeout=1800, # 30 minutes
cache_alias="llm_redis", # Routes to 'llm_redis' cache in settings.py
key_prefix="prod_v1", # Custom key namespace
)
Deterministic Hyperparameter Sensitivity
If responses vary based on sampling parameters like temperature or system_prompt, pass them as keyword arguments into .get(), .set(), or .generate_key():
# Keys generated will differ if temperature or system_prompt change
cached = cache.get(
prompt="Explain photosynthesis",
provider="openai",
model_name="gpt-4o",
temperature=0.2,
system_prompt="Explain for a 5-year old",
)
Manual Key Invalidation
# Invalidate a specific prompt entry
cache.delete("Explain photosynthesis", provider="openai", model_name="gpt-4o")
🧪 Running Tests
The test suite covers deterministic key hashing, error handling, lazy configuration checks, and cache routing.
# Clone the repository
git clone https://github.com/Hafiz-alt/django-llm-cache.git
cd django-llm-cache
# Run tests
python -m unittest discover -s tests -v
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
👤 Author & Support
- Author: Hafiz Muhammed P H
- Email: hafizmdph88172@gmail.com
- GitHub: @Hafiz-alt
- Bug Tracker: Issues
Metadata
Release files for django-llm-cache 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| django_llm_cache-0.1.0.tar.gz | 17.9 kB | Without provenance | Details for django_llm_cache-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| django_llm_cache-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for django_llm_cache-0.1.0-py3-none-any.whl |
Total release size: 28.6 kB
Release files / django_llm_cache-0.1.0.tar.gz
| Download URL | django_llm_cache-0.1.0.tar.gz |
|---|---|
| Size | 17.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e6b137cf5ab29d2535963fd7c8cfb2d0e5ae8a4b77edcbc4c68ec5d730e8ab95
|
|
BLAKE2b-256 checksum How to use checksums |
456dc8531b8b9113063a8475536845fa347ed27d3c71f1f8c23f8c1099b4a626
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Release files / django_llm_cache-0.1.0-py3-none-any.whl
| Download URL | django_llm_cache-0.1.0-py3-none-any.whl |
|---|---|
| Size | 10.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4209693430c0dbf5bcbdc02c70819e1c8a3ff534e93a4402c78379af3948b952
|
|
BLAKE2b-256 checksum How to use checksums |
cd00b791c4730e38f79d985fda986fe1867993bb6590d620d304d1759581a43b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.10
|
Security for django-llm-cache 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.