Make downloading scientific data much easier
Project description
data_downloader
Make downloading scientific data much easier
Introduction
data_downloader
is a very convenient and powerful data download package for retrieving files using HTTP/HTTPS. It current includes download model downloader
and url parsing model parse_urls
. As httpx
was used which provided a method to access website with synchronous and asynchronous way, you can download multiple files at the same time.
Highlight Features
data_downloader has several features to make retrieving files easy, including:
- Resumable: You can resume aborted downloads automatically when you re-execute the code if website support resuming (status code is 216 or 416 when send a HEAD request to the server supplying a Range header)
- Asynchronous: Can download multiple files at the same time when download a single file very slow.
- Convenient: Provide a easy way to manage your username and password and parse urls from different sources:
- netrc: Provide a convenient way to manage your username and password via
.netrc
file. You don't need to input your username and password every time when you download files from a website which requires authentication. See sections :ref:netrc
for more details - parse_urls: Provide various methods to parse urls from different sources. See sections :ref:
parse_urls
for more details
- netrc: Provide a convenient way to manage your username and password via
Installation
You can install data_downloader
via pip from PyPI:
pip install data_downloader
or you can install the latest version from GitHub:
pip install git+hhttps://github.com/Fanchengyan/data-downloader.git
Usage
The detailed documentation is available at: https://data-downloader.readthedocs.io/en/latest/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for data_downloader-1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 836e6c5d1aaeddb47fa120d7d2a9c53db9d192b7651a1f6fca033e5a875d3a8b |
|
MD5 | c6b44134086bc6223cbf39deb013cad4 |
|
BLAKE2b-256 | d24ba2722b95b5a98ef913216054db370e5a7fb44cf7912048d56cf0683f1f30 |