Skip to main content

A straightforward, dependency free way to update multiple progress bars with python's multiprocessing library.

Project description

pbar-pool

A straightforward, dependency free way to update multiple progress bars with python's multiprocessing library.

Specifically designed to work with multiprocessing.Pool - for which I found all other available solutions to be either overcomplicated, or completely broken.

Installation:

pip install pbar-pool

Usage:

from multiprocessing import Pool, cpu_count
from pbar_pool import PbarPool, Pbar
import time, random


def fn(x):
    for _ in Pbar(x, manager=pbars, name=f'Process {pbars.id()}', color=(255, 0, 0)):
        time.sleep(random.randint(0, 3))


to_process = [list(range(10)) for _ in range(100)]
pbars = PbarPool(width=100)

with Pool(processes=cpu_count(), initializer=pbars.initializer()) as p:
    global_pbar = Pbar(p.imap_unordered(fn, to_process), manager=pbars, name='global', total=len(to_process))
    for _ in global_pbar:
        pass

-->

global: 56/100 ████████████████████████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 56% 3.9156s/it 02:52 remaining
 Process 4: 5/10 ██████████████████████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 50% 1.4075s/it 00:07 remaining
 Process 2: 2/10 ████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 20% 1.5064s/it 00:12 remaining
 Process 3: 0/10 ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% 
 Process 1: 1/10 ██████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 10% 0.0003s/it 00:00 remaining

it's as simple as adding the initializer=pbars.initializer() option to your Pool object to give each Pool process access to the global progress bars pool.

Then, in each process, wrap your iterator in a Pbar() object, passing in the global PbarPool object as the manager argument.

You can also add a global progress bar to track the progress of the outer function, as seen in the example above.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pbar-pool-0.0.2.tar.gz (4.1 kB view details)

Uploaded Source

File details

Details for the file pbar-pool-0.0.2.tar.gz.

File metadata

  • Download URL: pbar-pool-0.0.2.tar.gz
  • Upload date:
  • Size: 4.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5

File hashes

Hashes for pbar-pool-0.0.2.tar.gz
Algorithm Hash digest
SHA256 74000e0ebf8152038b06e79ea49234df650eb281e0b92a77bb8f42544f71c201
MD5 52166c09a29aebdf3c71a922327878fd
BLAKE2b-256 9a0f691ecd7f293768774e6ee89604f0c65b70d835391a37ea04a95fcd4794a5

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page