Skip to main content

A simple and generic rate limiter to prevent too many function calls or http requests.

Project description

mThrottle

A simple and generic rate limiter to prevent too many function calls or http requests.

When making large number of HTTP requests, its essential to respect the API limits of a server.

The Throttle class ensures you do not exceed these limits.

Python version: >= 3.7

Install with PIP

pip install mthrottle

Usage

Throttle takes two arguments - a dictionary configuration and an integer maxPenaltyCount.

The max_penalty_count is the maximum number of 429 HTTP error code, allowed to be returned during a session. Exceeding this limit will result in a runtime error.

The configuration is a dictionary which defines the limits for each endpoint. Here, RPS stands for Requests Per Second and RPM stands for Requests Per Minute. Limits are set for each end point.

Configuration must have a default key which defines the default limits.

To rate limit, run th.check method before a request or function call.

from mthrottle import Throttle

config = {
    'default': {
        'rps': 3,
        'rpm': 150
    }
    'search': {
        'rps': 20
        'rpm': 500
    }
}

maxPenaltyCount = 10

th = Throttle(config, maxPenaltyCount)

for i in range(200):
    th.check()
    print(i, flush=True, end='\r' * 3)

th.check takes a key argument, matching one of the key values in config. If key is not specified, default is used.

Example using Throttle.penalize

Throttle.penalize returns a boolean and must be called when a 429 HTTP status code is returned. It returns True when maxPenaltyCount is exceeded.

from mthrottle import Throttle
import requests

config = {
    'run': {
        'rpm': 3
    }
}

maxPenaltyCount = 10

th = Throttle(config, maxPenaltyCount)

for i in range(200):
    th.check(key='run')
    r = requests.get('https://google.com')

    if r.status_code == 429:
        if th.penalize():
            raise RuntimeError('Too many API rate limit warnings.')

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mthrottle-0.0.2.tar.gz (16.2 kB view details)

Uploaded Source

Built Distribution

mthrottle-0.0.2-py3-none-any.whl (15.4 kB view details)

Uploaded Python 3

File details

Details for the file mthrottle-0.0.2.tar.gz.

File metadata

  • Download URL: mthrottle-0.0.2.tar.gz
  • Upload date:
  • Size: 16.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for mthrottle-0.0.2.tar.gz
Algorithm Hash digest
SHA256 c7e771e9fe02c9716e3d825c8a3f97859af693383c070175dd7a69e1a474115a
MD5 2758b2bca08ce34d68b60196d104a7bb
BLAKE2b-256 ae2ebf9010dc1b2f97ee83fa6f2dd82039dc7893463594218f20c83b3dc8892d

See more details on using hashes here.

File details

Details for the file mthrottle-0.0.2-py3-none-any.whl.

File metadata

  • Download URL: mthrottle-0.0.2-py3-none-any.whl
  • Upload date:
  • Size: 15.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for mthrottle-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 fc170ff29ec2e76ef57446328720e3c586dd82d9a83ccd79640c7b0f017b3a32
MD5 54277152c33b70d2921b1b9371a63b68
BLAKE2b-256 24680a695fc6fa882a4533401b79454fe5291dd27e7a6b6806cb25b883302171

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page