A smart async downloader with hybrid chunking strategy
Project description
PyFastDL (Async Smart Downloader) 🚀
A high-performance, asynchronous file downloader written in Python using aiohttp. It features a smart hybrid strategy that dynamically probes download speed to decide between single-chunk or multi-chunk downloading for optimal performance.
✨ Features
- 🧠 Smart Strategy: Probes the server speed first. If it's fast enough, it downloads in one go to save overhead. If slow, it switches to multi-threaded downloading.
- ⚡ Asynchronous: Built on
asyncioandaiohttpfor non-blocking I/O. - 🔌 Session Injection: Bring your own
aiohttp.ClientSession(pass cookies, auth headers, proxies, etc.). - ⏯ Resilient: Automatic retries for failed chunks.
- 📉 Speed Limiting: Optional bandwidth throttling.
- 💾 RAM Optimized: Buffers writes to disk to minimize I/O operations.
📦 Installation
pip install --upgrade pyfastdl
🚀 Usage
Basic Usage
import asyncio
from pyfastdl import SmartDownloader
async def main():
url = "https://link.testfile.org/100MB"
downloader = SmartDownloader(url, "file.zip", max_connections=16)
success = await downloader.download()
if success:
print("Download Complete!")
if __name__ == "__main__":
asyncio.run(main())
Advanced: Using Custom Session (Auth/Cookies)
import asyncio
import aiohttp
from pyfastdl import SmartDownloader
async def main():
headers = {"Authorization": "Bearer YOUR_TOKEN"}
async with aiohttp.ClientSession(headers=headers) as session:
downloader = SmartDownloader(
url="https://protected-server.com/file.zip",
output_file="secure_file.zip"
)
# Pass the session to the downloader
await downloader.download(session=session)
asyncio.run(main())
⚙️ Configuration
| Parameter | Default | Description |
|---|---|---|
max_connections |
16 | Max concurrent connections (chunks). |
min_split_size |
10MB | Minimum file size required to enable splitting. |
write_buff_size |
10MB | Size of data to buffer in RAM before writing to disk. |
max_speed_limit |
-1 | Limit download speed (bytes/s). -1 = Unlimited. |
progress_interval |
-1 | Interval (in seconds) to display download progress. -1 = Never. |
retry_attempts |
3 | Number of retries for a failed chunk. |
🤝 Contributing
Pull requests are welcome!
📝 License
MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pyfastdl-1.0.3.tar.gz.
File metadata
- Download URL: pyfastdl-1.0.3.tar.gz
- Upload date:
- Size: 8.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b71f75aa5664a5eca6c5bbbfd8edb4f94fce617614e0f8f2c01e832e866f70cb
|
|
| MD5 |
2f5fa381875e688cf244df666826351a
|
|
| BLAKE2b-256 |
4c4ec990302029da444088dfb30422e18fbcb08f1c2cbc02af52aa97a67060a0
|
File details
Details for the file pyfastdl-1.0.3-py3-none-any.whl.
File metadata
- Download URL: pyfastdl-1.0.3-py3-none-any.whl
- Upload date:
- Size: 8.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f01721ca6a08a8adf2a07cb4f162252c11cc47e8a8e62deaf48e23ea1c1c20cc
|
|
| MD5 |
dd5839de194562cde18a1a1d803d84a3
|
|
| BLAKE2b-256 |
fd938f1b4af2bb199fc40be597ec455ef53ea7ac5621040c309b2a8fec5275e2
|