Badger Batcher contains useful utilities for batching a sequence on records
Project description
Badger Batcher
Badger Batcher contains useful utilities for batching a sequence on records
Free software: MIT license
Documentation: https://badger-batcher.readthedocs.io.
Features
Split records based max limit for batch size:
>>> records = [f"record: {rec}" for rec in range(5)]
>>> batcher = Batcher(records, max_batch_size=2)
>>> batcher.batches()
[['record: 0', 'record: 1'], ['record: 2', 'record: 3'], ['record: 4']]
When processing big chunks of data, consider iterating instead:
>>> import sys
>>> records = (f"record: {rec}" for rec in range(sys.maxsize))
>>> batcher = Batcher(records, max_batch_size=2)
>>> for batch in batcher:
... first_batch = batch
... break
>>> first_batch
['record: 0', 'record: 1']
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.0 (2021-04-09)
First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
badger_batcher-0.2.3.tar.gz
(15.8 kB
view hashes)