cachemind 0.1.1

Semantic caching framework for LLMs

CacheMind is a lightweight semantic caching layer for LLM applications.

It helps reduce redundant LLM calls to improve response time and save LLM token costs.

📦Installation

pip install cachemind

⚡Quick Start

from cachemind import CacheMind
cache = CacheMind()

response = cache.query("What is artificial intelligence?")
print(response)

🧩Customization

CacheMind is designed to be flexible. You can plug in your own components for LLM, embedding, vector store, and policy.

Custom Embedding
class MyEmbedding:
def encode(self, text):
return [0.1] * 384
cache = CacheMind(embedding=MyEmbedding())

📊 Metrics

CacheMind also supports metrics tracking, allowing you to monitor cache performance and effectiveness over time.

stats = cache.metrics.get_stats()
print(stats)

Example output:
{
"total_queries": 10,
"cache_hits": 6,
"cache_misses": 4,
"hit_rate": 0.6,
"tokens_saved": 120,
"tokens_used": 80
}

License: MIT