Skip to main content

A contextmanager to track progress of joblib execution

Project description

joblib-progress

A contextmanager to track progress of joblib execution using rich.progress. joblib-progress

Why

The vanilla multiprocessing does not work when an object to multiprocess is not pickle-able. The joblib solves this, but then its progress is not tracked nicely. This library solves that tracking issue with joblib.

Install

> pip install joblib-progress

Usage

If you know the number of items

import time

from joblib import Parallel, delayed
from joblib_progress import joblib_progress


def slow_square(i):
    time.sleep(i / 2)
    return i ** 2

with joblib_progress("Calculating square...", total=10):
    Parallel(n_jobs=4)(delayed(slow_square)(number) for number in range(10))

If you don't know the number of items

with joblib_progress("Calculating square..."):
    Parallel(n_jobs=4)(delayed(slow_square)(number) for number in range(10))

Acknowledgments

The idea of using joblib.parallel.BatchCompletionCallBack is referenced from https://stackoverflow.com/a/58936697/5133167

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

joblib_progress-1.0.6.tar.gz (3.1 kB view details)

Uploaded Source

Built Distribution

joblib_progress-1.0.6-py3-none-any.whl (3.5 kB view details)

Uploaded Python 3

File details

Details for the file joblib_progress-1.0.6.tar.gz.

File metadata

  • Download URL: joblib_progress-1.0.6.tar.gz
  • Upload date:
  • Size: 3.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for joblib_progress-1.0.6.tar.gz
Algorithm Hash digest
SHA256 9b4d97f0ab0fc439be29e8d6ed3114b0a3140f60d709e8a6a6ed3633767e48bc
MD5 7eb34779830fab44744cc4e4e4c6001b
BLAKE2b-256 f565d6a5a6b322ffd4319bb39f22b0c0d71ad89233843764c6f1afc0c4b3b7aa

See more details on using hashes here.

Provenance

The following attestation bundles were made for joblib_progress-1.0.6.tar.gz:

Publisher: publish.yml on jonghwanhyeon/joblib-progress

Attestations:

File details

Details for the file joblib_progress-1.0.6-py3-none-any.whl.

File metadata

File hashes

Hashes for joblib_progress-1.0.6-py3-none-any.whl
Algorithm Hash digest
SHA256 fbd26706cb7fabf8c953dc5ca221ab3daa7e1daff6395e61655244e667bfa5a2
MD5 c8dcf54863d35947f39eb3d93a8f0549
BLAKE2b-256 cce4d71e71edc972d534f51cff72fa30edb8b0e5df109b2e5ae67a3985faebb9

See more details on using hashes here.

Provenance

The following attestation bundles were made for joblib_progress-1.0.6-py3-none-any.whl:

Publisher: publish.yml on jonghwanhyeon/joblib-progress

Attestations:

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page