batchqueue
This module provides an extension to the basic queue.Queue class in order to
handle situations where incoming data for the queue tends to come in bursts to
be processed as a group.
Example Usage:
def worker(queue=None):
self.assertIsNotNone(queue)
while True:
items = queue.get_batch()
for i in items:
# Do stuff
queue.task_done()
# Process a batch any time there's at least a 700ms lull in traffic
self.q = BatchQueue(lull_time=700)
# turn-on the worker thread
threading.Thread(target=worker, daemon=True, kwargs={'queue': self.q}).start()
# send ten task requests to the worker, pausing a random number of
# milliseconds (up to 1000)
for item in range(10):
sleep(random.randrange(0, 1000) / 1000.0)
self.q.put(item + 1)
# block until all tasks are done
self.q.join()
1 maintainer
Release files for batchqueue 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| batchqueue-0.1.0.tar.gz | 4.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| batchqueue-0.1.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 8.7 kB
Release files / batchqueue-0.1.0.tar.gz
| Download URL | batchqueue-0.1.0.tar.gz |
|---|---|
| Size | 4.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
63a89e167db445e91920ebb61857fd831501d2b149d9df9bf1cf7016509d3242
|
|
BLAKE2b-256 checksum How to use checksums |
7712fe3d2c672a53528e04806e6723fbdd82d27a478d1942661c81d01c09da81
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.5.0.1 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.6
|
Release files / batchqueue-0.1.0-py3-none-any.whl
| Download URL | batchqueue-0.1.0-py3-none-any.whl |
|---|---|
| Size | 4.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4de4e7c08f12f8a125606a2475e811fe75effb0b0fbac12ec0645630ec41300f
|
|
BLAKE2b-256 checksum How to use checksums |
d2ceafc5323db6839f84e9e385fdadc4a87bdf6674234db98d8af586282f6097
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.5.0.1 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.9.6
|
