Wrapper for tqdm and joblib to have a progressbar while batch processing
Project description
tqdm_batch
Batch processing using joblib including tqdm progress bars
Add batch processing to joblib, including tqdm progress bars.
Install
pip install tqdm_batch
Usage
Process a list of items
using a function
.
from tqdm_batch import batch_process
import random
import time
def batch_process_function(row, some_var):
time.sleep(0.01)
return row + random.randint(0, some_var)
N = 1_000
items = range(N)
result = batch_process(
items,
batch_process_function,
some_var=42,
n_workers=6,
sep_progress=True,
)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tqdm_batch-0.1.0.tar.gz
(4.2 kB
view hashes)
Built Distribution
Close
Hashes for tqdm_batch-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd5afc95698c401eb9b395ce216e9b818ee195feba9b2c6059ae707cb62f2d59 |
|
MD5 | e805bf8d5f72466d8a05569e4aa7cd96 |
|
BLAKE2b-256 | a152483432667a397a9ca38b3cc7c711690206038d8b5705efeb8fde161c0e9d |