Concurrent request HTTP execution library
Project description
Concurrent request HTTP execution library
What is it?
hyper-requests is a Python library that enables multithreading of API calls using asyncio. It takes a list of URLs and a list of parameters as input and then uses the requests library to make these calls asynchronously (https://pypi.org/project/requests/).
Usage
Installation
Install hyper-requests using pip:
pip install hyper-requests
Example
from hyper_requests.threader import AsyncRequests
# Define the request parameters
params = [
{'url': 'http://httpbin.org/get' , 'data': 'value1'},
{'url': 'http://httpbin.org/get' , 'data': 'value3'},
{'url': 'http://httpbin.org/get' , 'data': 'value5'},
{'url': 'http://httpbin.org/get' , 'data': 'value7'},
{'url': 'http://httpbin.org/get' , 'data': 'value9'}
]
# Create an instance of AsyncRequests and execute the requests
returned_data = AsyncRequests(request_params=params, workers=10).run_threads()
# Process the returned data
for response in returned_data:
print(response)
This example demonstrates the usage of hyper-requests to perform asynchronous HTTP requests.
First, make sure you have installed hyper-requests by running the command pip install hyper-requests.
Next, import the AsyncRequests class from the hyper_requests.threader module.
Create a list of request parameters using dictionaries, where each dictionary represents a set of parameters for an individual request. In this example, each request MUST have a URL specified with the 'url' key, all other paramters must match the classic request template.
Create an instance of AsyncRequests with the request_params argument set to your list of request parameters. Specify the number of concurrent worker threads to use with the workers argument (in this case, workers=10).
Execute the requests using the run_threads() method, which returns the data obtained from the requests.
Finally, process the returned data as desired. In this example, each response is printed, but you can perform further operations based on your specific needs.
Performance
It is hyper fast!
Within the test/performance
directory there is a performance test that makes 20 API calls to the random joke generator api: https://official-joke-api.appspot.com/random_joke.
Using hyper requests the time taken to make these calls is ~2 second, using syncronous api calls takes ~16 seconds.
============================= test session starts ==============================
collecting ... collected 1 item
test_performance.py::PerformanceTest::test_api_performance
============================== 1 passed in 17.76s ==============================
Process finished with exit code 0
PASSED [100%]
Asynchronous Execution time: 1.845513105392456 seconds
Asynchronous Data length: 20
Synchronous Execution time: 15.81911015510559 seconds
Synchronous Data length: 20
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hyper_requests-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39aec6e9e4b00cbddaa384194dab8dd045e644265373db41b254685074de519e |
|
MD5 | 563b1273c666c7ef4fd3af61408ddfa2 |
|
BLAKE2b-256 | 3cfd46c22c372213d331d91dbb730a022553e827acadea6bca46709b7497e999 |