flightlock

Single-flight cache stampede protection for Python.

When a cached value expires under concurrent load, most caching libraries let every waiting request independently recompute it — hammering your database or API with N identical calls at once (a "cache stampede" or "thundering herd"). flightlock ensures only one caller recomputes the value while every other caller waits for and reuses that result.

Named after the "singleflight" pattern used in production caching systems (e.g. Go's golang.org/x/sync/singleflight, groupcache).

Table of contents

• Why flightlock
• Quickstart
• Benchmark
• How it works
• Features
• Installation
• Comparison
• Roadmap
• Contributing
• Security
• Support
• License

Why flightlock

Picture a cache key with a 60-second TTL backing an expensive DB query. The moment it expires, if 500 concurrent requests arrive for that key before it's repopulated, a naive cache sends all 500 straight to your database at once. flightlock coordinates those 500 callers so exactly one hits the database, and the other 499 wait milliseconds for that one result instead.

Quickstart

from flightlock import cached

@cached(ttl=60)
def get_user(user_id: int) -> dict:
    print("hitting the database...")
    return expensive_db_call(user_id)

get_user(1)  # prints "hitting the database...", takes 300ms
get_user(1)  # instant — cache hit, no print

With Redis and metrics enabled:

from flightlock import cached
from flightlock.backends.redis import RedisBackend

backend = RedisBackend(host="localhost", port=6379)

@cached(ttl=60, backend=backend, metrics=True)
def get_user(user_id: int) -> dict:
    return expensive_db_call(user_id)

Benchmark

Run it yourself: python benchmarks/stampede_benchmark.py

Simulating 200 concurrent requests for the same cache key, with a 0.3s simulated origin latency (e.g. a slow DB query):

Result: 9.5% fewer origin calls with flightlock (200 -> 1 calls for 200 concurrent requests)

Every request still returns the correct value — flightlock just ensures only one caller does the work while the other 199 wait for and share the result.

How it works

1. A per-key lock registry tracks in-flight computations.
2. When a caller arrives for a missing/expired key, it either becomes the leader (first arrival — runs the function) or a follower (waits for the leader's result).
3. The leader runs the function outside the lock, so unrelated keys never block each other.
4. When the leader finishes, every waiting follower is woken simultaneously and receives the same result (or the same exception, if it failed).

See src/flightlock/core.py for the implementation.

Features

• ✅ Single-flight stampede protection (in-process)
• ✅ Pluggable backends: in-memory, Redis
• ✅ TTL jitter to prevent synchronized mass-expiry across many keys
• ✅ Prometheus-compatible metrics (hits, misses, errors, latency)
• ✅ Custom cache key functions
• 🔜 Async support
• 🔜 Cross-process distributed locking (currently: stampede protection is per-process; Redis backend shares storage across processes, not the lock)

Installation

pip install flightlock                  # core, zero dependencies
pip install flightlock[redis]           # + Redis backend
pip install flightlock[metrics]         # + Prometheus metrics
pip install flightlock[redis,metrics]   # everything

Comparison with alternatives

flightlock isn't trying to replace general-purpose caching libraries — it solves one specific, real production problem (concurrent cache-miss stampedes) thoroughly.

Roadmap

• Project scaffolding, CI, packaging config
• Core single-flight lock implementation
• In-memory backend
• The @cached() decorator
• TTL jitter
• Redis backend
• Metrics hook
• Benchmark proving stampede protection under load
• Publish to PyPI
• Async support

Contributing

Contributions are welcome. See CONTRIBUTING.md for setup instructions and guidelines.

Security

See SECURITY.md for how to report a vulnerability.

Support

See SUPPORT.md for how to get help.

AI-assisted development

This project was built with AI pair-programming assistance for scaffolding, boilerplate, and code review. See AI_POLICY.md for details on how AI was used and what was independently written, tested, and verified.

License

MIT — see LICENSE.