hypercache
Explicit, persistent caching for expensive Python functions and methods.
What it does
- Caches expensive calls (API calls, embeddings, LLM generations)
- Works with sync and async methods
- Persists across restarts (disk, extensible to Redis)
- Normalizes non-hashable inputs (dicts, Pydantic models, dataclasses, bytes)
- Supports TTL, stale windows, and background refresh
Why not functools.lru_cache or cachetools
| lru_cache | cachetools | hypercache | |
|---|---|---|---|
| Async support | No | No | Yes |
| Persistent storage | No | No | Yes |
| Non-hashable inputs | No | No | Yes (normalize) |
| Instance state in key | N/A | Manual | config= |
| TTL / stale / refresh | No | TTL only | Yes |
Install
pip install hypercache
Observe cache telemetry
Any library can observe cache decisions by installing a scoped callback:
from hypercache import CachePolicy, CacheService, MemoryStore, observe_cache
cache = CacheService(MemoryStore())
events = []
with observe_cache(events.append):
cache.run(
instance="demo",
operation="embed",
version="embed:v1",
inputs={"text": "hello"},
policy=CachePolicy(),
compute=lambda: {"vector": [1, 2, 3]},
)
event = events[0]
assert event.hit is False
assert event.operation == "embed"
The observer is task-local via ContextVar, so nested async calls stay scoped to
the current request or workflow run.
Basic usage
from datetime import timedelta
from hypercache import CachePolicy, CacheService, MemoryStore, cached
def _embedder_config(self) -> dict:
return {"model": self.model, "dimensions": self.dimensions}
class Embedder:
def __init__(self, model: str = "text-embedding-3-large", dimensions: int = 1536):
self._cache = CacheService(MemoryStore(max_entries=512))
self.model = model
self.dimensions = dimensions
@cached(
version="embed:v1",
policy=CachePolicy(
ttl=timedelta(hours=6),
stale=timedelta(minutes=30),
refresh_in_background=True,
),
config=_embedder_config,
)
async def embed(self, text: str) -> dict:
return await call_embedding_api(text)
- Inputs are auto-captured from the function signature. No duplicate parameter lists.
config=explicitly declares which instance state affects the cache key. No hidden method lookups.version=lets you invalidate all cached values when the implementation changes.
Sharing config across methods
Define the config function once, reference it from multiple decorators:
def _llm_config(self) -> dict:
return {"model": self.model, "temperature": self.temperature}
class LLM:
def __init__(self, model: str, temperature: float):
self._cache = CacheService(MemoryStore())
self.model = model
self.temperature = temperature
@cached(version="generate:v1", policy=CachePolicy(), config=_llm_config)
async def generate(self, prompt: str) -> dict:
...
@cached(version="structured:v1", policy=CachePolicy(), config=_llm_config)
async def generate_structured(self, prompt: str, schema: dict) -> dict:
...
Excluding inputs from the key
Use exclude= to drop arguments that shouldn't affect caching:
@cached(
version="embed:v1",
policy=CachePolicy(),
config=_embedder_config,
exclude=frozenset({"request_id", "trace_id"}),
)
async def embed(self, text: str, request_id: str | None = None, trace_id: str | None = None):
...
Persistent cache
Swap the store — everything else stays the same:
from pathlib import Path
from hypercache import DiskCacheStore
cache = CacheService(DiskCacheStore(Path("./cache")))
Direct usage (no decorator)
result = cache.run(
instance=embedder,
operation="embed",
version="embed:v1",
inputs={"text": "hello"},
config={"model": embedder.model},
policy=CachePolicy(),
compute=lambda: embedder.embed_uncached("hello"),
)
Invalidation
Embedder.embed.key_for(embedder, "hello") # inspect key
Embedder.embed.invalidate(embedder, "hello") # delete one entry
Embedder.embed.clear(embedder) # delete all entries for this method
Design principles
- No magic: no hidden method lookups, no Protocols that silently match by name
- Explicit:
config=in the decorator, not a convention on the class - DRY: inputs auto-captured from signature, no duplicate parameter lists
- IDE-friendly: named functions, not lambdas; errors surface at import time
Metadata
Release files for hypercache 0.2.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| hypercache-0.2.2.tar.gz | 10.2 kB | Without provenance | Details for hypercache-0.2.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| hypercache-0.2.2-py3-none-any.whl | Python 3 | none | any | Details for hypercache-0.2.2-py3-none-any.whl |
Total release size: 23.1 kB
Release files / hypercache-0.2.2.tar.gz
| Download URL | hypercache-0.2.2.tar.gz |
|---|---|
| Size | 10.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
583e47f865585d4cc6be7b96c0530fa468bb6d755e15e8c337f911551bd85369
|
|
BLAKE2b-256 checksum How to use checksums |
5b453f05f427c62006362ff691449c71abbc27740d6b22394322826a2303531d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
Release files / hypercache-0.2.2-py3-none-any.whl
| Download URL | hypercache-0.2.2-py3-none-any.whl |
|---|---|
| Size | 12.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c3044ae79a3bb7021d46f0ce2302a86476a241f78a0e57225283727922f2bc8a
|
|
BLAKE2b-256 checksum How to use checksums |
1ce8d9d81d62b487e367c90c1c2c3ba4ce9f69c6e4862c760c1d9a9df2ff5306
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
Security for hypercache 0.2.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.