Skip to main content

Simple, easy-to-use throttler for asyncio

Project description

asyncio-throttle

travis-ci pypi-version

Simple, easy-to-use throttler for asyncio.

Example

import time
import random
import asyncio

from asyncio_throttle import Throttler

async def worker(no, throttler, n):
    for _ in range(n):
        await asyncio.sleep(random.random() * 2)

        async with throttler:
            print(time.time(), 'Worker #%d: Bang!' % no)

async def main():
    throttler = Throttler(rate_limit=5)

    tasks = [
        loop.create_task(worker(no, throttler, 10))
            for no in range(5)
    ]
    await asyncio.wait(tasks)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

Here I limited work rate to 5/sec while there are 5 workers. And the result:

1508273760.3462772 Worker #2: Bang!
1508273760.590009 Worker #3: Bang!
1508273760.856431 Worker #0: Bang!
1508273761.0110679 Worker #2: Bang!
1508273761.086856 Worker #4: Bang!
1508273761.350699 Worker #3: Bang!
1508273761.5906 Worker #1: Bang!
1508273761.8655958 Worker #4: Bang!
1508273762.224158 Worker #0: Bang!
1508273762.600234 Worker #2: Bang!
1508273762.694332 Worker #2: Bang!
1508273762.726774 Worker #0: Bang!
1508273762.944273 Worker #4: Bang!

Installation

$ pip install asyncio-throttle

Usage

asyncio_throttle.Throttler introduces simple APIs: flush() and acquire(). But you will not be interested in those because you can just use it within with statement and it looks nicer.

First, create a throttler given desired rate limit. For example if you want to limit rate to 500/min, you can make it as:

from asyncio_throttle import Throttler

throttler = Throttler(rate_limit=500, period=60)

Then whenever you want to do some jobs which should have limited rate(e.g. sending request to server), Put it in async with statement:

async with throttler:
    send_a_request()

It's that easy. asyncio_throttler can be easily integrated with aiohttp too:

async def worker(throttler, session):
    while True:
        async with throttler:
            async with session.get('http://example.com') as resp:
                do_some_job_with(await resp.text())

        await asyncio.sleep(0.05)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

asyncio-throttle-1.0.0.tar.gz (3.6 kB view details)

Uploaded Source

Built Distribution

asyncio_throttle-1.0.0-py3-none-any.whl (4.1 kB view details)

Uploaded Python 3

File details

Details for the file asyncio-throttle-1.0.0.tar.gz.

File metadata

  • Download URL: asyncio-throttle-1.0.0.tar.gz
  • Upload date:
  • Size: 3.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.7.7

File hashes

Hashes for asyncio-throttle-1.0.0.tar.gz
Algorithm Hash digest
SHA256 8a9199b5bd4844c19bb8474d0f0df23962fb2fe837ae35fde1d501ae3942ac5a
MD5 52f24c4e32a92b0662d689c2fbfffffd
BLAKE2b-256 507263ce49d984fd5d87cffc81dee634c6d0664bfe08d5a1bf72d53bd5d4dcc8

See more details on using hashes here.

File details

Details for the file asyncio_throttle-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: asyncio_throttle-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 4.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.44.1 CPython/3.7.7

File hashes

Hashes for asyncio_throttle-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ac284c65e6d595687f8b4025bdb6f81a012fbc640eb76258f65d39fadcb5dc8b
MD5 914b42cb21c61f59bb7fbabc9e0eefe6
BLAKE2b-256 00a287e28afc7ff726dfcf1615357d8a261a37dae48888479c093fa4b856085d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page