Skip to main content

A smart async downloader with hybrid chunking strategy

Project description

PyFastDL (Async Smart Downloader) 🚀

A high-performance, asynchronous file downloader written in Python using aiohttp. It features a smart hybrid strategy that dynamically probes download speed to decide between single-chunk or multi-chunk downloading for optimal performance.

✨ Features

  • 🧠 Smart Strategy: Probes the server speed first. If it's fast enough, it downloads in one go to save overhead. If slow, it switches to multi-threaded downloading.
  • ⚡ Asynchronous: Built on asyncio and aiohttp for non-blocking I/O.
  • 🔌 Session Injection: Bring your own aiohttp.ClientSession (pass cookies, auth headers, proxies, etc.).
  • ⏯ Resilient: Automatic retries for failed chunks.
  • 📉 Speed Limiting: Optional bandwidth throttling.
  • 💾 RAM Optimized: Buffers writes to disk to minimize I/O operations.

📦 Installation

pip install --upgrade pyfastdl

🚀 Usage

Basic Usage

import asyncio
from pyfastdl import SmartDownloader

async def main():
    url = "https://link.testfile.org/100MB"
    downloader = SmartDownloader(url, "file.zip", max_connections=16)
    
    success = await downloader.download()
    if success:
        print("Download Complete!")

if __name__ == "__main__":
    asyncio.run(main())

Advanced: Using Custom Session (Auth/Cookies)

import asyncio
import aiohttp
from pyfastdl import SmartDownloader

async def main():
    headers = {"Authorization": "Bearer YOUR_TOKEN"}
    
    async with aiohttp.ClientSession(headers=headers) as session:
        downloader = SmartDownloader(
            url="https://protected-server.com/file.zip",
            output_file="secure_file.zip"
        )
        # Pass the session to the downloader
        await downloader.download(session=session)

asyncio.run(main())

⚙️ Configuration

Parameter Default Description
max_connections 16 Max concurrent connections (chunks).
min_split_size 10MB Minimum file size required to enable splitting.
write_buff_size 10MB Size of data to buffer in RAM before writing to disk.
max_speed_limit -1 Limit download speed (bytes/s). -1 = Unlimited.
progress_interval -1 Interval (in seconds) to display download progress. -1 = Never.
retry_attempts 3 Number of retries for a failed chunk.

🤝 Contributing

Pull requests are welcome!

📝 License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyfastdl-1.0.3.tar.gz (8.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pyfastdl-1.0.3-py3-none-any.whl (8.5 kB view details)

Uploaded Python 3

File details

Details for the file pyfastdl-1.0.3.tar.gz.

File metadata

  • Download URL: pyfastdl-1.0.3.tar.gz
  • Upload date:
  • Size: 8.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.12.2

File hashes

Hashes for pyfastdl-1.0.3.tar.gz
Algorithm Hash digest
SHA256 b71f75aa5664a5eca6c5bbbfd8edb4f94fce617614e0f8f2c01e832e866f70cb
MD5 2f5fa381875e688cf244df666826351a
BLAKE2b-256 4c4ec990302029da444088dfb30422e18fbcb08f1c2cbc02af52aa97a67060a0

See more details on using hashes here.

File details

Details for the file pyfastdl-1.0.3-py3-none-any.whl.

File metadata

  • Download URL: pyfastdl-1.0.3-py3-none-any.whl
  • Upload date:
  • Size: 8.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.12.2

File hashes

Hashes for pyfastdl-1.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 f01721ca6a08a8adf2a07cb4f162252c11cc47e8a8e62deaf48e23ea1c1c20cc
MD5 dd5839de194562cde18a1a1d803d84a3
BLAKE2b-256 fd938f1b4af2bb199fc40be597ec455ef53ea7ac5621040c309b2a8fec5275e2

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page