np-stream
This Python package provides a replacement for the np.load(), np.save() and np.savez_compressed() NumPy functions with support for streaming data on one dimension.
For uncompressed arrays with a known size, use np.memmap() instead.
Installation
$ pip install np-stream
Usage
Writing uncompressed data
The dimension which lists entries is added first for C-contiguous arrays (the default when creating arrays) and last otherwise. In this example, the array is C-contiguous, therefore its final shape is (100, 6, 2).
from np_stream import NpyStreamWriter
import numpy as np
with NpyStreamWriter(file) as writer:
for index in range(100):
writer.write(np.random.rand(6, 2))
Alternatively, the writer can be used without a context manager. The call to flush() at the end is mandatory.
writer = NpyStreamWriter(file, shape=entry_shape)
for index in range(100):
writer.write(np.random.rand(6, 2))
writer.flush()
Writing compressed data
To produced compressed data, just replace NpyStreamWriter with NpzStreamWriter and create an .npz instead of an .npy file. The writer uses Deflate compression.
- from np_stream import NpyStreamWriter
+ from np_stream import NpzStreamWriter
entry_shape = (6, 2)
- with NpyStreamWriter(file, shape=entry_shape) as writer:
+ with NpzStreamWriter(file, shape=entry_shape) as writer:
for index in range(100):
writer.write(np.random.rand(*entry_shape))
Reading uncompressed data
reader = NpyStreamReader(file)
for entry in reader:
print(entry)
Reading compressed data
reader = NpzStreamReader(file)["arr_0"]
for entry in reader:
print(entry)
Similar packages
Metadata
Release files for np-stream 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| np-stream-0.1.0.tar.gz | 8.0 kB | Without provenance | Details for np-stream-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| np_stream-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for np_stream-0.1.0-py3-none-any.whl |
Total release size: 17.6 kB
Release files / np-stream-0.1.0.tar.gz
| Download URL | np-stream-0.1.0.tar.gz |
|---|---|
| Size | 8.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5f64d175877325a3ce3b51ebe976a828bfb0638590d64242a033f30f5086eaff
|
|
BLAKE2b-256 checksum How to use checksums |
7fd4542e445605da74edf85a0869a3b65dc4eaf351a192ae3fa7e53b32c127ed
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.2
|
Release files / np_stream-0.1.0-py3-none-any.whl
| Download URL | np_stream-0.1.0-py3-none-any.whl |
|---|---|
| Size | 9.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
983913e70e7935f366c1ed6f5168fa554e1d76692204a6a51097649995a0adab
|
|
BLAKE2b-256 checksum How to use checksums |
2986d44f00fb26f8f26280c48cd4cb593bc5febb50faa8026c33e761f3fbc9f3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.2
|
Security for np-stream 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.