Semantic caching wrapper for Redis — stop paying for duplicate LLM calls

These details have not been verified by PyPI

Project links

Project description

semantic-redis-cache

Semantic caching for Redis. Stop paying for duplicate LLM calls.

The Problem

Traditional caching misses similar queries:

"what is monthly revenue?"  → cache miss ❌
"show me monthly revenue"   → cache miss ❌
"monthly revenue figures?"  → cache miss ❌

All three mean the same thing. You pay for 3 LLM calls instead of 1.

The Solution

semantic-redis-cache uses sentence embeddings to find semantically similar cached responses. Zero cost embeddings — runs locally on your machine.

"what is monthly revenue?"  → cache miss  → calls LLM → stores result
"show me monthly revenue"   → cache HIT ✅ → returns cached result
"monthly revenue figures?"  → cache HIT ✅ → returns cached result

Install

pip install semantic-redis-cache

Requirements

Redis running locally or remotely
Python 3.8+

# Run Redis locally with Docker
docker run -d -p 6379:6379 redis

Quick Start

from semcache import SemanticRedisCache

# initialize cache
cache = SemanticRedisCache(threshold=0.85)

# define your LLM function
def call_llm(question: str) -> str:
    # replace with your actual LLM call
    # works with OpenAI, Claude, Groq, Ollama — anything
    return "Your LLM response here"

# get_or_set — checks cache first, calls LLM only on miss
result = cache.get_or_set(
    query="what is monthly revenue?",
    func=call_llm,
    question="what is monthly revenue?"
)

print(result.value)      # LLM response
print(result.hit)        # False (first call — cache miss)
print(result.similarity) # 0.0

# second call — similar question
result = cache.get_or_set(
    query="show me monthly revenue",
    func=call_llm,
    question="show me monthly revenue"
)

print(result.value)      # same cached response
print(result.hit)        # True  ✅ cache hit
print(result.similarity) # 0.91  ← similarity score

Full Example — With OpenAI

from semcache import SemanticRedisCache
from openai import OpenAI

client = OpenAI()
cache = SemanticRedisCache(threshold=0.85)

def call_openai(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": question}]
    )
    return response.choices[0].message.content

questions = [
    "what is monthly revenue?",
    "show me the monthly revenue",   # similar → cache hit
    "monthly revenue figures?",      # similar → cache hit
    "what is the weather today?",    # different → cache miss
]

for question in questions:
    result = cache.get_or_set(
        query=question,
        func=call_openai,
        question=question
    )
    status = "HIT" if result.hit else "MISS"
    print(f"[{status}] ({result.similarity:.2f}) {question}")

print(cache.metrics)

Output:

[MISS] (0.00) what is monthly revenue?
[HIT]  (0.94) show me the monthly revenue
[HIT]  (0.91) monthly revenue figures?
[MISS] (0.12) what is the weather today?

{
  'hits': 2,
  'misses': 2,
  'hit_rate': '50.0%',
  'total_cached': 2
}

Full Example — With Claude

from semcache import SemanticRedisCache
import anthropic

client = anthropic.Anthropic()
cache = SemanticRedisCache(threshold=0.85)

def call_claude(question: str) -> str:
    message = client.messages.create(
        model="claude-opus-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": question}]
    )
    return message.content[0].text

result = cache.get_or_set(
    query="explain semantic caching",
    func=call_claude,
    question="explain semantic caching"
)

print(result.value)
print(result.hit)

Full Example — With Ollama (Free, Local)

from semcache import SemanticRedisCache
import ollama

cache = SemanticRedisCache(threshold=0.85)

def call_ollama(question: str) -> str:
    response = ollama.chat(
        model="llama3",
        messages=[{"role": "user", "content": question}]
    )
    return response["message"]["content"]

result = cache.get_or_set(
    query="what is machine learning?",
    func=call_ollama,
    question="what is machine learning?"
)

print(result.value)
print(result.hit)

Test Script — No LLM Required

Run this to verify everything is working:

from semcache import SemanticRedisCache

# initialize
cache = SemanticRedisCache(
    host="localhost",
    port=6379,
    threshold=0.85,
    namespace="test"
)

# clear any existing test data
cache.flush()

# mock LLM function — no API key needed
call_count = 0
def mock_llm(question: str) -> str:
    global call_count
    call_count += 1
    return f"Answer to: {question}"

print("=" * 50)
print("semantic-redis-cache test")
print("=" * 50)

# test 1 — cache miss (first call)
result = cache.get_or_set(
    "what is monthly revenue?",
    mock_llm,
    question="what is monthly revenue?"
)
print(f"\nTest 1 — First call:")
print(f"  Hit:        {result.hit}")        # False
print(f"  LLM calls:  {call_count}")        # 1
print(f"  Value:      {result.value}")

# test 2 — semantic hit (similar question)
result = cache.get_or_set(
    "show me the monthly revenue",
    mock_llm,
    question="show me the monthly revenue"
)
print(f"\nTest 2 — Similar question:")
print(f"  Hit:        {result.hit}")        # True
print(f"  Similarity: {result.similarity}") # ~0.90
print(f"  LLM calls:  {call_count}")        # still 1 — not called again

# test 3 — cache miss (unrelated question)
result = cache.get_or_set(
    "what is the weather today?",
    mock_llm,
    question="what is the weather today?"
)
print(f"\nTest 3 — Unrelated question:")
print(f"  Hit:        {result.hit}")        # False
print(f"  LLM calls:  {call_count}")        # 2

# test 4 — metrics
print(f"\nMetrics:")
print(f"  {cache.metrics}")

# test 5 — manual get and set
cache.set("capital of France?", "Paris")
result = cache.get("what is the capital of France?")
print(f"\nTest 4 — Manual get/set:")
print(f"  Hit:        {result.hit}")        # True
print(f"  Value:      {result.value}")      # Paris
print(f"  Similarity: {result.similarity}")

print("\n" + "=" * 50)
print("All tests passed ✅")
print("=" * 50)

Expected output:

==================================================
semantic-redis-cache test
==================================================

Test 1 — First call:
  Hit:        False
  LLM calls:  1
  Value:      Answer to: what is monthly revenue?

Test 2 — Similar question:
  Hit:        True
  Similarity: 0.9034
  LLM calls:  1

Test 3 — Unrelated question:
  Hit:        False
  LLM calls:  2

Metrics:
  {'hits': 1, 'misses': 2, 'hit_rate': '33.33%', 'total_cached': 2}

Test 4 — Manual get/set:
  Hit:        True
  Value:      Paris
  Similarity: 0.9187

==================================================
All tests passed ✅
==================================================

Configuration

cache = SemanticRedisCache(
    host="localhost",                # Redis host
    port=6379,                       # Redis port
    threshold=0.85,                  # similarity threshold (0.0 - 1.0)
    ttl=86400,                       # cache TTL in seconds (24 hours)
    model_name="all-MiniLM-L6-v2",  # sentence transformer model
    namespace="semcache"             # Redis key namespace
)

Threshold Guide

Threshold	Behaviour
0.70	Aggressive — high hit rate, risk of wrong answers
0.85	Balanced — recommended starting point ✅
0.90	Conservative — safe, fewer hits
0.95	Strict — almost like exact matching

API Reference