An optionally asynchronous multithreaded downloader for python
Project description
multithread
Multithread is an optionally asynchronous Python library for downloading files using several threads.
Features
- Lightweight: one file, a little over 100 lines of code excluding license
- Extensive: the ability to pass your own sessions and your own arguments to each request means you don't need to wait for your desired feature to be implemented; anything you can do in aiohttp, you can do in multithread!
- Fast: benefit from the speed of aiohttp and multithreaded downloading!
Installation
Requirements:
- Python 3.5.3+
- aiohttp
- mosquito/aiofile
- optional: tqdm
Use the package manager pip to install multithread.
For support for progress bars, install multithread[progress].
pip3 install multithread
Usage
import multithread
download_object = multithread.Download("http://url.com/file", filename)
download_object.start()
# passing headers (you can pass any other arguments that aiohttp.ClientSession.request can take as well)
download_object = multithread.Download("http://url.com/file", filename, aiohttp_args={"headers": {"a": "b", "c": "d"}})
download_object.start()
Documentation
Downloader
Downloader(self, url, file, threads=4, session=None, progress_bar=True, aiohttp_args={'method': 'GET'}, create_dir=True)
A multi-threaded downloader class using aiohttp
Attributes:
- url (str): The URL to download
- file (str, path-like object or AIOFile): The file to write the download to. if not an AIOFile, it's used as a path for one
- threads (int): The number of threads to use to download
- session (aiohttp.ClientSession): An existing session to use with aiohttp
- new_session (bool): True if a session was not passed, and the downloader created a new one
- progress_bar (bool): Whether to output a progress bar or not
- aiohttp_args (dict): Arguments to be passed in each aiohttp request. If you supply a Range header using this, it will be overwritten in fetch()
start
Downloader.start(self)
Calls asyncstart() synchronously
asyncstart
Downloader.asyncstart(self)
Re-initializes file and calls download() with it. Closes session if necessary
fetch
Downloader.fetch(self, fileobj, progress=False, filerange=(0, ''))
Individual thread for fetching files.
Parameters:
- fileobj (AIOFile): the file to write the download to
- progress (bool or tqdm.Progress): the progress bar (or lack thereof) to update
- filerange (tuple): the range of the file to get
download
Downloader.download(self, fileobj)
Generates ranges and calls fetch() with them.
Parameters:
- fileobj (AIOFile): the file to write the download to
Contributing
Any and all pull requests are welcome. As this is a small project, there are no strict standards, but please try to keep your code clean to a reasonable standard. Alternatively, if you would like to clean my code, that would be more than welcome!
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for multithread-1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c28803a6329abad283981101d473b697f95dd53a6a294175ef018ffd7354f81 |
|
MD5 | 59684068d6c68b3dada0a30f5c527967 |
|
BLAKE2b-256 | 57d3871fa7d078a956a1d0bed4e74e883ada7147d3fd4c5ef0415c8b9028efa4 |