A contextmanager to track progress of joblib execution
Project description
joblib-progress
A contextmanager to track progress of joblib
execution using rich.progress
.
Why
The vanilla multiprocessing
does not work when an object to multiprocess is not pickle-able
. The joblib
solves this, but then its progress is not tracked nicely. This library solves that tracking issue with joblib
.
Install
> pip install joblib-progress
Usage
If you know the number of items
import time
from joblib import Parallel, delayed
from joblib_progress import joblib_progress
def slow_square(i):
time.sleep(i / 2)
return i ** 2
with joblib_progress("Calculating square...", total=10):
Parallel(n_jobs=4)(delayed(slow_square)(number) for number in range(10))
If you don't know the number of items
with joblib_progress("Calculating square..."):
Parallel(n_jobs=4)(delayed(slow_square)(number) for number in range(10))
Acknowledgments
The idea of using joblib.parallel.BatchCompletionCallBack
is referenced from https://stackoverflow.com/a/58936697/5133167
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file joblib_progress-1.0.6.tar.gz
.
File metadata
- Download URL: joblib_progress-1.0.6.tar.gz
- Upload date:
- Size: 3.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b4d97f0ab0fc439be29e8d6ed3114b0a3140f60d709e8a6a6ed3633767e48bc |
|
MD5 | 7eb34779830fab44744cc4e4e4c6001b |
|
BLAKE2b-256 | f565d6a5a6b322ffd4319bb39f22b0c0d71ad89233843764c6f1afc0c4b3b7aa |
Provenance
The following attestation bundles were made for joblib_progress-1.0.6.tar.gz
:
Publisher:
publish.yml
on jonghwanhyeon/joblib-progress
-
Statement type:
https://in-toto.io/Statement/v1
- Predicate type:
https://docs.pypi.org/attestations/publish/v1
- Subject name:
joblib_progress-1.0.6.tar.gz
- Subject digest:
9b4d97f0ab0fc439be29e8d6ed3114b0a3140f60d709e8a6a6ed3633767e48bc
- Sigstore transparency entry: 147298742
- Sigstore integration time:
- Predicate type:
File details
Details for the file joblib_progress-1.0.6-py3-none-any.whl
.
File metadata
- Download URL: joblib_progress-1.0.6-py3-none-any.whl
- Upload date:
- Size: 3.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fbd26706cb7fabf8c953dc5ca221ab3daa7e1daff6395e61655244e667bfa5a2 |
|
MD5 | c8dcf54863d35947f39eb3d93a8f0549 |
|
BLAKE2b-256 | cce4d71e71edc972d534f51cff72fa30edb8b0e5df109b2e5ae67a3985faebb9 |
Provenance
The following attestation bundles were made for joblib_progress-1.0.6-py3-none-any.whl
:
Publisher:
publish.yml
on jonghwanhyeon/joblib-progress
-
Statement type:
https://in-toto.io/Statement/v1
- Predicate type:
https://docs.pypi.org/attestations/publish/v1
- Subject name:
joblib_progress-1.0.6-py3-none-any.whl
- Subject digest:
fbd26706cb7fabf8c953dc5ca221ab3daa7e1daff6395e61655244e667bfa5a2
- Sigstore transparency entry: 147298743
- Sigstore integration time:
- Predicate type: