rust-py-cache

An ultra-fast local cache for Python, powered by Rust.

A local, in-memory, thread-safe cache with TTL, lazy expiration, and metrics. The core is written in Rust (PyO3 + maturin) on top of a concurrent DashMap; the Python API is minimal. Think of it as a "mini Redis" living inside your Python process.

🌐 Website: rust-py-cache.vercel.app

Installation

pip install rust-py-cache

To work on it locally (requires Rust + maturin):

python -m venv .venv && source .venv/bin/activate
pip install maturin pytest
maturin develop          # compiles the Rust core and installs into the venv
pytest                   # runs the tests

Usage

from rust_py_cache import Cache

cache = Cache()

cache.set("user:1", {"name": "Roberto"}, ttl=60)   # ttl in seconds
user = cache.get("user:1")                          # {"name": "Roberto"}
cache.get("missing", default=0)                     # 0

cache.exists("user:1")        # True (honors TTL)
cache.delete("user:1")        # True if removed, False if absent
cache.len()                   # approximate size
cache.keys()                  # list of keys
cache.cleanup_expired()       # remove expired entries; returns the count
cache.clear()                 # remove everything (keeps counters)
cache.stats()                 # {'hits','misses','sets','deletes','expired','evicted','size'}

Bounded cache with LRU eviction

# Cap the number of keys. When full and a new key arrives, evict the
# least-recently-used entries (a ~3% batch) instead of rejecting the write.
cache = Cache(max_size=1000, eviction_policy="lru")
cache.eviction_policy            # "lru"

# Default policy is "reject": set() returns False when full (and the key is new).
cache = Cache(max_size=1000)     # eviction_policy="reject"
cache.set("a", 1)                # True / False

Background expiration

# A background thread reclaims expired entries every N seconds, so you don't
# have to call cleanup_expired() yourself. It stops when the cache is collected.
cache = Cache(cleanup_interval=30)   # seconds (int/float)

Memoization decorator

@cache.cached(ttl=60)
def add(a, b):
    return a + b

add(2, 3)   # runs and caches
add(2, 3)   # served from cache

# custom key (fixed string or callable):
@cache.cached(ttl=300, key=lambda user_id: f"user:{user_id}")
def load_user(user_id):
    ...

See full examples under examples/ (FastAPI and Django).

API

Constructor: Cache(max_size=None, eviction_policy="reject", cleanup_interval=None). eviction_policy must be "reject" or "lru" (any other value raises ValueError). cleanup_interval (seconds, > 0) enables the background sweeper.

How it works

• Serialization: in the MVP, values are serialized with pickle (on the Python side, via PyO3) and stored as opaque bytes (Vec<u8>) in the Rust core.
• Concurrency: DashMap (a HashMap with per-shard locks) plus AtomicU64 counters, with no global lock on the hot path. Thread-safe, no busy loop.
• TTL: expiration is lazy by default — an expired key is removed when accessed (get/exists) or via cleanup_expired(). Pass cleanup_interval to also run a background sweeper thread that reclaims expired keys on its own.
• Eviction: with max_size + eviction_policy="lru", a full cache evicts the least-recently-used entries (recency updated on every get hit) to admit a new key. It removes them in a batch of max_size / 32 (at least 1) per pass, so the O(n) scan that finds them runs once every ~n/32 inserts instead of on every one — amortized O(1) per write, with no global lock on the get/set path. A set on a full 100k-entry cache went from ~620 µs (one full scan per insert, up to 0.2.x) to ~1.6 µs. Right after a pass the cache sits ~3% below max_size.

Limitations

• The cache is process-local: multiple workers = multiple independent caches.
• It does not replace Redis for distributed caching.
• Data is lost when the process restarts.
• pickle must not be used to deserialize untrusted data.
• Lazy TTL by default: without cleanup_interval, expired items may linger until accessed or until cleanup_expired() runs.

Development

cargo test          # Rust core tests
maturin develop     # rebuild and install
pytest              # Python tests

If maturin develop complains about both VIRTUAL_ENV and CONDA_PREFIX being set, run conda deactivate first, or use env -u CONDA_PREFIX maturin develop.

Roadmap

Stages and next steps (LRU/LFU eviction, background expiration, configurable serializer, namespaces, etc.) are in ROADMAP.md.

License

MIT