High-Performance Asynchronous HTTP Client setting Requests on Fire 🔥
Project description
FireRequests 🔥
FireRequests is a high-performance, asynchronous HTTP client library for Python, engineered to accelerate your file transfers. By harnessing advanced concepts like semaphores, exponential backoff with jitter, concurrency, and fault tolerance, FireRequests can achieve up to a 5x real-world speedup in file downloads and uploads compared to traditional synchronous methods.
Features 🚀
- Asynchronous I/O: Non-blocking network and file operations using
asyncio
,aiohttp
, andaiofiles
, boosting throughput for I/O-bound tasks. - Concurrent Transfers: Uses
asyncio.Semaphore
to limit simultaneous tasks, optimizing performance by managing system resources effectively. - Fault Tolerance: Retries failed tasks with exponentially increasing wait times, adding random jitter to prevent network congestion.
- Chunked Processing: Files are split into configurable chunks for parallel processing, significantly accelerating uploads/downloads.
- Compatibility: Supports environments like Jupyter through
nest_asyncio
, enabling reusableasyncio
loops for both batch and interactive Jupyter use.
Installation 📦
Install FireRequests using pip:
pip install firerequests
Quick Start 🏁
Accelerate your downloads with just a few lines of code:
from firerequests import FireRequests
url = "https://example.com/largefile.iso"
filename = "largefile.iso"
fr = FireRequests()
fr.download(url, filename)
Real-World Speed Test 🏎️
FireRequests delivers significant performance improvements over traditional download methods. Below is the result of a real-world speed test:
Normal Download 🐌: 100%|██████████| 3.42G/3.42G [06:16<00:00, 9.08MB/s]
Downloading on 🔥: 100%|██████████| 3.42G/3.42G [01:15<00:00, 45.2MB/s]
🐌 Download Time: 376.77 seconds
🔥 Download Time: 75.75 seconds
Advanced Usage ⚙️
Downloading Files
from firerequests import FireRequests
url = "https://example.com/largefile.iso"
filename = "largefile.iso"
fr = FireRequests()
fr.download(url, filename, max_files=10, chunk_size=2 * 1024 * 1024)
url
: The URL of the file to download.filename
: The local filename to save the downloaded file.max_files
: The maximum number of concurrent chunk downloads.chunk_size
: The size of each chunk in bytes.
Uploading Files
from firerequests import FireRequests
file_path = "largefile.iso"
parts_urls = ["https://example.com/upload_part1", "https://example.com/upload_part2", ...]
fr = FireRequests()
fr.upload(file_path, parts_urls, chunk_size=2 * 1024 * 1024, max_files=10)
Comparing Download Speed
from firerequests import FireRequests
url = "https://example.com/largefile.iso"
filename = "largefile.iso"
fr = FireRequests()
fr.compare_speed(url, filename)
License 📄
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for firerequests-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9e7bc71bdc48c9354961f01787c341776d2f33167949f5e689c20b94ecc27379 |
|
MD5 | 2a0476c293781aaea6ba3a45561920d9 |
|
BLAKE2b-256 | 684342deb0c0aadae9757a7bb43d7404da57cae3d7b5a2b95b51612a3e8bb56d |