A straightforward, dependency free way to update multiple progress bars with python's multiprocessing library.
Project description
pbar-pool
A straightforward, dependency free way to update multiple progress bars with python's multiprocessing library.
Specifically designed to work with multiprocessing.Pool - for which I found all other available solutions to be either overcomplicated, or completely broken.
Installation:
pip install pbar-pool
Usage:
from multiprocessing import Pool, cpu_count
from pbar_pool import PbarPool, Pbar
import time, random
def fn(x):
for _ in Pbar(x, manager=pbars, name=f'Process {pbars.id()}', color=(255, 0, 0)):
time.sleep(random.randint(0, 3))
to_process = [list(range(10)) for _ in range(100)]
pbars = PbarPool(width=100)
with Pool(processes=cpu_count(), initializer=pbars.initializer()) as p:
global_pbar = Pbar(p.imap_unordered(fn, to_process), manager=pbars, name='global', total=len(to_process))
for _ in global_pbar:
pass
-->
global: 56/100 ████████████████████████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 56% 3.9156s/it 02:52 remaining
Process 4: 5/10 ██████████████████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 50% 1.4075s/it 00:07 remaining
Process 2: 2/10 ████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 20% 1.5064s/it 00:12 remaining
Process 3: 0/10 ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0%
Process 1: 1/10 ██████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 10% 0.0003s/it 00:00 remaining
it's as simple as adding the initializer=pbars.initializer()
option to your Pool object to give each Pool process access to the global progress bars pool.
Then, in each process, wrap your iterator in a Pbar()
object, passing in the global PbarPool object as the manager
argument.
You can also add a global progress bar to track the progress of the outer function, as seen in the example above.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file pbar-pool-0.0.2.tar.gz
.
File metadata
- Download URL: pbar-pool-0.0.2.tar.gz
- Upload date:
- Size: 4.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 74000e0ebf8152038b06e79ea49234df650eb281e0b92a77bb8f42544f71c201 |
|
MD5 | 52166c09a29aebdf3c71a922327878fd |
|
BLAKE2b-256 | 9a0f691ecd7f293768774e6ee89604f0c65b70d835391a37ea04a95fcd4794a5 |