langchain-redis-rate-limiter 0.1.0

Redis-based distributed RateLimiter for LangChain with RPM/RPS support

LangChain Redis Rate Limiter

English | 한국어

A Redis-based RateLimiter for LangChain, compatible with BaseRateLimiter. It supports both Requests Per Minute (RPM) and Requests Per Second (RPS) limiting, designed to work seamlessly in distributed environments.

Features

• Redis Support: Compatible with Local Redis, AWS ElastiCache.
• LangChain Integration: Inherits from BaseRateLimiter, allowing direct usage with LangChain models.
• Async Support: Supports both sync and async request.
• Distributed Support: Uses Redis to manage shared status across multiple servers or processes.

Installation

pip install langchain-redis-rate-limiter

Usage

Direct Model Integration (Recommended)

You can pass the RedisRateLimiter directly to LangChain models that support the rate_limiter parameter (e.g., ChatOpenAI, ChatAnthropic, ChatGoogleGenerativeAI).

from langchain_redis_rate_limiter import RedisRateLimiter
from langchain_openai import ChatOpenAI

# 1 request per second (60 RPM)
limiter = RedisRateLimiter(
    redis_url="redis://localhost:6379",
    requests_per_second=1,
    check_every_n_seconds=0.1,
    max_bucket_size=1
)

model = ChatOpenAI(
    model="gpt-4o",
    rate_limiter=limiter
)

model.invoke("Hello world")

Parameters

• redis_url: Redis connection URL (e.g., redis://localhost:6379).
• key_prefix: Prefix for Redis keys (default: langchain_limiter). Use different prefixes for different limit rules.
• requests_per_second: The maximum number of requests allowed per second. For example, setting this to 5 means you can make up to five requests each second.
• check_every_n_seconds: When the limit is reached and a request needs to wait, this value determines how often the system checks again to see whether a new request is allowed. Example: 0.1 checks every 0.1 seconds.
• max_bucket_size: Controls how many requests can be handled in a short burst. A larger value allows brief spikes in traffic before the limiter slows things down.

License

Apache-2.0 license