Parallel Media Downloader

Helps you to download media file in parallel without async / await syntax.

Feature

This project helps you to download media files in parallel without async / await syntax.

Installation

pip install parallelmediadownloader

Usage

Minimum example:

from datetime import datetime

from parallelmediadownloader.media_download_coroutine import DownloadOrder
from parallelmediadownloader.media_file import SaveOrder
from parallelmediadownloader.parallel_media_downloader import ParallelMediaDownloader

path_directory_download = "path/directory/download"
created_date_time = datetime.now()
list_download_order = [
    DownloadOrder(
        "https://example.com/test01.png",
        SaveOrder(
            path_directory_download,
            "test01.png",
            created_date_time,
        ),
    ),
    DownloadOrder(
        "https://example.com/test02.png",
        SaveOrder(
            path_directory_download,
            "test02.png",
            created_date_time,
        ),
    ),
    DownloadOrder(
        "https://example.com/test03.png",
        SaveOrder(
            path_directory_download,
            "test03.png",
            created_date_time,
        ),
    ),
]
list_media_download_result = ParallelMediaDownloader.execute(list_download_order)

API

ParallelMediaDownloader.execute

class ParallelMediaDownloader:
    """API of parallel media downloading."""

    @staticmethod
    def execute(
        list_download_order: Iterable[DownloadOrder],
        *,
        limit: int = 5,
        media_filter: Optional[MediaFilter] = None,
        allow_http_status: List[int] = None
    ) -> List[MediaDownloadResult]:

list_download_order: Iterable[DownloadOrder]

List of DownloadOrder. Method will download them in parallel.

limit: int = 5

Limit number of parallel processes.

media_filter: Optional[MediaFilter] = None

Filter extends MediaFilter to remove downloaded media file depending on file or content of media. NotImageFilter will be help to understand its roll:

class NotImageFilter(MediaFilter):
    def _filter(self, media_file: MediaFile) -> bool:
        return not str(media_file.path_file).lower().endswith((".png", ".jpg", ".jpeg", ".gif"))

allow_http_status: List[int] = None

By default, ParallelMediaDownloader.execute will check HTTP status code by Response.raise_for_status and whole process will stop. When HTTP status applies allow_http_status, process will continue. Then, MediaDownloadResult.media_file will be None.