Downloaders

Python package to handle the download of multiple types of files.

How do I install this package?

As usual, just download it using pip:

pip install downloaders

Usage examples

from downloaders import BaseDownloader

downloader = BaseDownloader()
urls = [...]
downloader.download(urls)

Troubleshooting

MacOS multiprocessing nightmare fuel

Cupertino has a gift for us: somehow, under certain specific astral configurations on MacOS (which I have yet to fully understand), multiprocessing will crash with the following error:

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Apparently, this can be easily fixed by changing the way multiprocessing spawns processes:

import platform, multiprocessing

if platform.system() == "Darwin":
    multiprocessing.set_start_method('spawn')

The aforementioned solution was proposed on this StackOverflow question.