A simple and generic rate limiter to prevent too many function calls or http requests.
Project description
mThrottle
A simple and generic rate limiter to prevent too many function calls or http requests.
When making large number of HTTP requests, its essential to respect the API limits of a server.
The Throttle class ensures you do not exceed these limits.
Python version: >= 3.7
Install with PIP
pip install mthrottle
Usage
Throttle takes two arguments - a dictionary configuration
and an integer maxPenaltyCount
.
The max_penalty_count
is the maximum number of 429 HTTP error code, allowed to be returned during a session. Exceeding this limit will result in a runtime error.
The configuration
is a dictionary which defines the limits for each endpoint. Here, RPS
stands for Requests Per Second and RPM
stands for Requests Per Minute. Limits are set for each end point.
Configuration must have a default
key which defines the default limits.
To rate limit, run th.check
method before a request or function call.
from mthrottle import Throttle
config = {
'default': {
'rps': 3,
'rpm': 150
}
'search': {
'rps': 20
'rpm': 500
}
}
maxPenaltyCount = 10
th = Throttle(config, maxPenaltyCount)
for i in range(200):
th.check()
print(i, flush=True, end='\r' * 3)
th.check
takes a key
argument, matching one of the key values in config. If key is not specified, default
is used.
Example using Throttle.penalize
Throttle.penalize
returns a boolean and must be called when a 429 HTTP status code is returned. It returns True when maxPenaltyCount
is exceeded.
from mthrottle import Throttle
import requests
config = {
'run': {
'rpm': 3
}
}
maxPenaltyCount = 10
th = Throttle(config, maxPenaltyCount)
for i in range(200):
th.check(key='run')
r = requests.get('https://google.com')
if r.status_code == 429:
if th.penalize():
raise RuntimeError('Too many API rate limit warnings.')
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file mthrottle-0.0.2.tar.gz
.
File metadata
- Download URL: mthrottle-0.0.2.tar.gz
- Upload date:
- Size: 16.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7e771e9fe02c9716e3d825c8a3f97859af693383c070175dd7a69e1a474115a |
|
MD5 | 2758b2bca08ce34d68b60196d104a7bb |
|
BLAKE2b-256 | ae2ebf9010dc1b2f97ee83fa6f2dd82039dc7893463594218f20c83b3dc8892d |
File details
Details for the file mthrottle-0.0.2-py3-none-any.whl
.
File metadata
- Download URL: mthrottle-0.0.2-py3-none-any.whl
- Upload date:
- Size: 15.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fc170ff29ec2e76ef57446328720e3c586dd82d9a83ccd79640c7b0f017b3a32 |
|
MD5 | 54277152c33b70d2921b1b9371a63b68 |
|
BLAKE2b-256 | 24680a695fc6fa882a4533401b79454fe5291dd27e7a6b6806cb25b883302171 |