Python HTTP requests throttler
Project description
RequestsThrottler is an Apache2 Licensed HTTP library, written in Python, and powered by futures and Requests. See the full documentation at http://pythonhosted.org/RequestsThrottler.
With RequestsThrottler you can easily throttle HTTP requests! After having created your throttler with a delay of your choice, you just have to:
Start the throttler
Submit your requests
Shutdown the throttler
Here’s an example:
import requests from requests_throttler import BaseThrottler bt = BaseThrottler(name='base-throttler', delay=1.5) request = requests.Request(method='GET', url='http://www.google.com') reqs = [request for i in range(0, 5)] bt.start() throttled_requests = bt.multi_submit(reqs) bt.shutdown() responses = [tr.response for tr in throttled_requests]
Too hard? Then just submit your requests inside a with statement! Here’s an example:
import requests from requests_throttler import BaseThrottler with BaseThrottler(name='base-throttler', delay=1.5) as bt: request = requests.Request(method='GET', url='http://www.google.com') reqs = [request for i in range(0, 5)] throttled_requests = bt.multi_submit(reqs) responses = [tr.response for tr in throttled_requests]
Installation
Use pip to install RequestsThrottler:
>>> pip install RequestsThrottler
Features
BaseThrottler a simple throttler with a fixed amount of delay
Release History
0.2.1 (2014-01-14)
Added implicit way to set delay for BaseThrottler by using reqs_over_time
0.2.0 (2014-01-04)
Reorganized modules
Changed submit method signature in favor of two methods: submit and multi_submit
Added documentation
0.1.1 (2013-12-31)
Fixed not working previous release
Changed example.py
0.1.0 (2013-12-30)
BaseThrottler a simple throttler with a fixed amount of delay
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.