Skip to main content

Python package to handle download of multiple types of files.

Project description

Pypi project Pypi total project downloads Github Actions

Python package to handle the download of multiple types of files.

How do I install this package?

As usual, just download it using pip:

pip install downloaders

Usage examples

from downloaders import BaseDownloader

downloader = BaseDownloader()
urls = [...]
downloader.download(urls)

Troubleshooting

MacOS multiprocessing nightmare fuel

Cupertino has a gift for us: somehow multiprocessing on MacOS in some specific astral configurations that I have yet to fully understand, it will crash with the following error:

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Apparently, this can be easily fixed by changing the way multiprocessing spawns processes, that is:

import platform, multiprocessing

if platform.system() == "Darwin":
    multiprocessing.set_start_method('spawn')

The aforementioned solution was proposed on this StackOverflow question.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

downloaders-1.0.19.tar.gz (10.6 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page