throttlecore

A production-grade, highly resilient distributed rate limiting package for Python applications utilizing Redis.

throttlecore implements atomic evaluation strategies using single-threaded Redis Lua scripts to eliminate network round-trips and race conditions in multi-node environments.

🌟 Features

• 🚀 Atomic Operations: All logic executes inside Redis memory, eliminating race conditions.
• ⚙️ Pluggable Strategies: Supports both Sliding Window Log (eliminating edge-of-window bursts) and Token Bucket (handling bursts with smooth refills).
• 🔒 FastAPI Native: Comes with first-class decorator and dependency injection support.
• 🛡️ Operational Resilience (Fail-Open): If your central Redis cluster goes offline, throttlecore defaults to a fail-open policy to keep your app responsive, warning operators without blocking users.
• 📊 Simulation Dashboard: Includes a premium, responsive glassmorphic web dashboard to visually inspect window sliding, token refilling, and throughput graphs in real time.
• 🧪 Zero-Config Local Testing: Out-of-the-box support for fakeredis so you can write tests and run the simulator without starting a real Redis instance.

🏗️ Architecture

sequenceDiagram
    participant Client
    participant FastAPI Node A
    participant FastAPI Node B
    participant Redis (Atomic Engine)

    Client->>FastAPI Node A: Request /api/data (X-API-Key: key123)
    Note over FastAPI Node A: Executes throttlecore middleware
    FastAPI Node A->>Redis: EVALSHA Sliding Window Script (key123)
    Note over Redis: Drops logs older than (Now - Window)<br/>Counts current logs in ZSET<br/>Admit request & updates TTL if under limit
    Redis-->>FastAPI Node A: [Allowed: true, Remaining: 4, RetryAfter: 0]
    FastAPI Node A-->>Client: HTTP 200 OK (Allowed)

    Client->>FastAPI Node B: Concurrent Request /api/data (X-API-Key: key123)
    FastAPI Node B->>Redis: EVALSHA Sliding Window Script (key123)
    Note over Redis: Log limit exceeded
    Redis-->>FastAPI Node B: [Allowed: false, Remaining: 0, RetryAfter: 4.85]
    FastAPI Node B-->>Client: HTTP 429 Too Many Requests (Retry-After: 5)

📦 Installation

To install throttlecore with core dependencies:

pip install redis fastapi

To install with development and simulation dashboard dependencies (including fakeredis and uvicorn):

pip install "throttlecore[dev]"
# Or locally
pip install -e .[dev]

🚀 Quick Start

1. Basic Python Usage

import asyncio
import redis.asyncio as aioredis
from throttlecore import DistributedRateLimiter, TokenBucketStrategy, RateLimitExceeded

async def main():
    # 1. Initialize Redis Client
    redis_client = aioredis.from_url("redis://localhost:6379/0", decode_responses=True)
    
    # 2. Initialize Rate Limiter (Defaults to Sliding Window Log Strategy)
    limiter = DistributedRateLimiter(
        redis_client=redis_client,
        default_strategy=TokenBucketStrategy(), # Switch strategies easily
        fail_open=True                          # Fail-open if Redis goes offline
    )
    
    # 3. Check and assert limits
    try:
        # Allow 5 requests per 10 seconds
        await limiter.is_allowed(key="user_12345", limit=5, window_seconds=10)
        print("Request allowed!")
    except RateLimitExceeded as e:
        print(f"Blocked! Try again in {e.retry_after:.2f} seconds.")
        
    await redis_client.aclose()

asyncio.run(main())

2. Integration with FastAPI

Inject rate limiting directly into path operations or globally using standard FastAPI dependencies:

from fastapi import FastAPI, Depends
import redis.asyncio as aioredis
from throttlecore import DistributedRateLimiter, RateLimitDecorator, TokenBucketStrategy

app = FastAPI()
redis_client = aioredis.from_url("redis://localhost:6379/0", decode_responses=True)
limiter = DistributedRateLimiter(redis_client)

# Define a route limited to 5 requests per 10 seconds (Sliding Window Log)
@app.get("/items", dependencies=[Depends(RateLimitDecorator(limiter, limit=5, window=10))])
async def read_items():
    return [{"item_id": "Foo"}, {"item_id": "Bar"}]

# Define a route limited using Token Bucket Strategy and consuming 2 tokens per request
@app.get("/heavy-job", dependencies=[
    Depends(RateLimitDecorator(
        limiter=limiter,
        limit=10,
        window=60,
        cost=2,
        strategy=TokenBucketStrategy()
    ))
])
async def run_heavy_job():
    return {"status": "processing"}

📊 Live Simulation Dashboard

throttlecore comes bundled with a beautiful visual simulator. It runs on fakeredis by default if no running Redis instance is detected on localhost:6379, so it works instantly.

To run the simulator:

python -m throttlecore.dashboard.app

Then navigate to http://localhost:8000.

Dashboard Visualizations:

• Sliding Window Log Timeline: Watch request timestamps sliding along a timeline in real-time, falling off the window boundary as time moves forward.
• Token Bucket Liquid Gauge: A dynamic container displaying the fluid level (current token count) refilling smoothly between request bursts.
• Live Charts: Charts plotting accepted and blocked throughput per second.
• Redis DB Inspector: Live JSON display showing what is stored in Redis (ZSET records or Hash fields) at any millisecond.

🧪 Testing

We use pytest with pytest-asyncio and fakeredis[lua] to validate all components, concurrency race conditions, and algorithm variations.

# Run the test suite
pytest tests/ -v

🛡️ License

Distributed under the MIT License. See LICENSE for details.