arraystream
Iterator-style transformations for numeric arrays, similar to itertools but optimized for array.array and buffer-protocol objects.
Overview
arraystream provides composable, allocation-aware transformations for numeric arrays, complementing arrayops which focuses on fast, Rust-backed numeric kernels.
Mental model: itertools, but for numeric arrays and memory buffers.
Where arrayops provides the execution engine (numeric kernels), arraystream provides the transformation language (structural & stream transforms).
Features
- Operate on
array.arrayand buffer-protocol objects - Favor zero-copy views where possible
- Avoid Python scalar iteration
- Small, predictable, and fast
- Composes naturally with
arrayops - No dependency on NumPy or Pandas
Installation
pip install arraystream
For development:
git clone https://github.com/eddiethedean/arraystream
cd arraystream
pip install -r requirements-dev.txt
maturin develop
Quick Start
import array
from arraystream import windowed, scan, diff
# Create an array
data = array.array('i', [1, 2, 3, 4, 5, 6, 7, 8])
# Sliding windows
for window in windowed(data, size=3):
print(list(window)) # [1, 2, 3], [2, 3, 4], ...
# Differences between consecutive elements
diffs = diff(data)
print(list(diffs)) # [1, 1, 1, 1, 1, 1, 1]
# Prefix sum (scan)
prefix_sums = scan(data, op="sum")
print(list(prefix_sums)) # [1, 3, 6, 10, 15, 21, 28, 36]
Relationship to arrayops
| arrayops | arraystream |
|---|---|
| Numeric kernels | Structural & stream transforms |
| In-place math | Shape-changing operations |
| Minimal API | Rich combinators |
| Low-level | Expressive |
arrayops acts as the execution engine
arraystream acts as the transformation language
API Categories
Structural Transforms
Operations that rearrange or expose structure without computing new values:
chunked(arr, size)- Return views of chunkswindowed(arr, size, step=1)- Sliding windowstake(arr, n)- First n elements (view)drop(arr, n)- Skip first n elements (view)interleave(a, b)- Interleave two arraysrepeat_each(arr, n)- Repeat each element n times
Numeric Stream Operations
Operations that compute new values:
scan(arr, op="sum")- Prefix reducediff(arr)- Differences between consecutive elementspairwise(arr)- Return pairs of consecutive elementsclip(arr, min, max)- Clip values to range
Boolean & Index Operations
where(arr, predicate)- Filter elements by predicateargwhere(arr)- Return indices where condition is truemask(arr, mask_array)- Apply boolean mask
Grouping & Segmentation
run_length_encode(arr)- Run-length encodinggroupby_runs(arr)- Group consecutive equal elementssegment_by(arr, boundaries)- Split at boundary indices
Supported Types
Supports the same 10 typecodes as arrayops:
b- signed charB- unsigned charh- signed shortH- unsigned shorti- signed intI- unsigned intl- signed longL- unsigned longf- floatd- double
Performance
- Structural operations: Python + buffer protocol (zero-copy views)
- Numeric transforms: Rust (PyO3) for performance
- Zero-copy wherever semantics allow
- Explicit copies when safety or clarity requires them
Use Cases
- Streaming numeric pipelines
- Signal processing
- Sensor data preprocessing
- Embedded or constrained environments
- ETL micro-kernels
- Teaching numeric programming without NumPy
Development
# Run tests
pytest tests/ -v
# Run with coverage
pytest tests/ --cov=arraystream --cov-report=html
# Format code
ruff format .
# Lint code
ruff check .
# Type checking
mypy arraystream tests
License
MIT License - see LICENSE file for details.
Acknowledgments
Metadata
Release files for arraystream 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| arraystream-0.1.0.tar.gz | 20.9 kB | Without provenance | Details for arraystream-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| arraystream-0.1.0-cp38-cp38-macosx_11_0_arm64.whl | CPython 3.8 | CPython 3.8 | macOS 11.0+ ARM64 | Details for arraystream-0.1.0-cp38-cp38-macosx_11_0_arm64.whl |
Total release size: 268.5 kB
Release files / arraystream-0.1.0.tar.gz
| Download URL | arraystream-0.1.0.tar.gz |
|---|---|
| Size | 20.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e0318489dee852412302099ff63656c54bd635fd2df7a56e282f3787d0e89dd8
|
|
BLAKE2b-256 checksum How to use checksums |
7132cf9dd824c4831e3df3481908b1c37060073aca7026bb476bff6d1a16167a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.18
|
Release files / arraystream-0.1.0-cp38-cp38-macosx_11_0_arm64.whl
| Download URL | arraystream-0.1.0-cp38-cp38-macosx_11_0_arm64.whl |
|---|---|
| Size | 247.6 kB |
| Tags | CPython 3.8 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
abb2c03890773ba746f285eab90c84ada44efbb3cf122cab199fc82ddb857b1b
|
|
BLAKE2b-256 checksum How to use checksums |
42dcf95d04702594f5da83e95f0c6e082261423533a4f69b6cc80ee46c83dc45
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.8.18
|
Security for arraystream 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.