ArrayQueues
This package provides a drop-in replacement for the Python multiprocessing Queue class which handles transport of large numpy arrays. It avoids pickling and uses the multiprocessing Array class in the background. The major difference between this implementation and the normal queue is that the maximal amount of memory that the queue can have must be specified beforehand.
Attempting to send an array of a different shape or datatype of the previously inserted one resets the queue. Only passing of numpy arrays is supported, optionally annotated with timestamps if using the TimestampedArrayQueue class, but other object types can be supported by extending the class.
The package has been tested on Python 3.6/3/7 on Windows and MacOS and Linux with Travis. Python 2.7 is not supported.
Usage example
from arrayqueues.shared_arrays import ArrayQueue
from multiprocessing import Process
import numpy as np
class ReadProcess(Process):
def __init__(self, source_queue):
super().__init__()
self.source_queue = source_queue
def run(self):
print(self.source_queue.get())
if __name__ == "__main__":
q = ArrayQueue(1) # intitialises an ArrayQueue which can hold 1MB of data
n = np.full((5,5), 5)
q.put(n)
r = ReadProcess(q)
r.start()
r.join()
Further examples can be found in tests.
Metadata
Release files for arrayqueues 1.4.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| arrayqueues-1.4.1.tar.gz | 8.6 kB | Without provenance | Details for arrayqueues-1.4.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| arrayqueues-1.4.1-py3-none-any.whl | Python 3 | none | any | Details for arrayqueues-1.4.1-py3-none-any.whl |
Total release size: 15.0 kB
Release files / arrayqueues-1.4.1.tar.gz
| Download URL | arrayqueues-1.4.1.tar.gz |
|---|---|
| Size | 8.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ec8fb90503bf82cbd34440e405fc6b31b970dc93df638f12e0a2383c218fb456
|
|
BLAKE2b-256 checksum How to use checksums |
92e8839da8946cfc4f31b758ed4eb5f51e9b189922f801f83b3a89f8b5202d77
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.4
|
Release files / arrayqueues-1.4.1-py3-none-any.whl
| Download URL | arrayqueues-1.4.1-py3-none-any.whl |
|---|---|
| Size | 6.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ea0a5a5e99e0c61dc92f9fc36e29c728b20d68494d51b05578b3217890c7e41d
|
|
BLAKE2b-256 checksum How to use checksums |
032d9d5407501f61521e79449c8dcbb23b6bf38ff10ee9f2323e557e465dadae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.4
|
Security for arrayqueues 1.4.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.