Bytewax
Bytewax is an open source Python framework for building highly scalable dataflows in a streaming or batch context.
Dataflow programming is a programming paradigm where program execution is conceptualized as data flowing through a series of operations or transformations.
At a high level, Bytewax provides a few major benefits:
- The operators in Bytewax are largely “data-parallel”, meaning they can operate on independent parts of the data concurrently.
- The ability to express higher-level control constructs, like iteration.
- Bytewax allows you to develop and run your code locally, and then easily scale that code to multiple workers or processes without changes.
Bytewax uses PyO3 to provide Python bindings to the Timely Dataflow Rust library.
Visit our documentation site for full documentation
Usage
Install the latest release with pip:
pip install bytewax
Example
Here is an example of a simple dataflow program using Bytewax:
# pyexamples/simple.py
from bytewax import Executor
ec = Executor()
flow = ec.Dataflow(enumerate(range(10)))
flow.map(lambda x: x * x)
flow.inspect(print)
if __name__ == "__main__":
ec.build_and_run()
Running the program:
python ./pyexamples/simple.py
0
1
4
9
16
25
36
49
81
64
For a more complete example, and documentation on the available operators, check out the User Guide.
License
Bytewax is licensed under the Apache-2.0 license.
1 maintainer
Metadata
1 maintainer
Release files for bytewax 0.7.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| bytewax-0.7.1.tar.gz | 588.3 kB | Details |
Built distributions (wheels)
Total release size: 29.9 MB
Release files / bytewax-0.7.1.tar.gz
| Download URL | bytewax-0.7.1.tar.gz |
|---|---|
| Size | 588.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
460334c4bc0b5ca6f2db515af7c4de9c59b150302dad87cef197fffe066dbe25
|
|
BLAKE2b-256 checksum How to use checksums |
a49656232bfd4bfda731526133b3c4d1b67000ece97a1bacd94614a08aed5561
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
| Download URL | bytewax-0.7.1-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl |
|---|---|
| Size | 3.9 MB |
| Tags | Linux glibc 2.12+ x86-64 PyPy 3.7 PyPy 3.7 7.3 |
|
SHA-256 checksum How to use checksums |
f157463e2b2a67b7fa9afdc49d913bc6a17f4762c943457974e4d6ca08fa371f
|
|
BLAKE2b-256 checksum How to use checksums |
dfc38e042bd28b67c8a0f204dc5124aeb467833f1048c600b9a22a153e81deee
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp310-none-win_amd64.whl
| Download URL | bytewax-0.7.1-cp310-none-win_amd64.whl |
|---|---|
| Size | 618.1 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
5ba7f286a1e66c5d388b2d6f57f0f7ca55094b82c6f9dcf6df85101834327530
|
|
BLAKE2b-256 checksum How to use checksums |
ffa11c7a714e3b21cc2677d5591c7fdae6c18b2fb17326592b7fdce4bb64ff8f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
| Download URL | bytewax-0.7.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl |
|---|---|
| Size | 3.9 MB |
| Tags | CPython 3.10 Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
d023230293fad5a8a449a61961b23dc6044c00202b4e045cb629f90826697ab4
|
|
BLAKE2b-256 checksum How to use checksums |
876c4c6497938913314c421c237f55fa85aeccd5e0e853f2756b59894558d99d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
| Download URL | bytewax-0.7.1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
92f0df3b4687c19194d9e44ad15fbde643fc705acb3defb47a576fd51e10289f
|
|
BLAKE2b-256 checksum How to use checksums |
437030901dd0f40fcce847d94cc4fd0722cb3fe51a79dfb5913f374639f638c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp39-none-win_amd64.whl
| Download URL | bytewax-0.7.1-cp39-none-win_amd64.whl |
|---|---|
| Size | 619.3 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
99dec37e1de9f964ec9ed05605342b3e8b47a2789853e827a7b6e2677eb504a8
|
|
BLAKE2b-256 checksum How to use checksums |
3a7b2abb14d81e05b689888a640a6e57068e2326856b8d766e61ca72ccb99963
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
| Download URL | bytewax-0.7.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.9 Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
db7758b9fce2ee12ed25f7f3bd310b60721c3ee709300454813893b4e265ecef
|
|
BLAKE2b-256 checksum How to use checksums |
c8cc50573aff781cca537aee367dc086d759c9226c0761e1a2d96037df0f8d83
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
| Download URL | bytewax-0.7.1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
560a58fa5ec70fe36d103bc6fb6236913f8a4c1e93d1fe6d04fbd4ae3dbd4e1f
|
|
BLAKE2b-256 checksum How to use checksums |
69f0198125cc0730f733f298eb90a84013c1e8d6b44827551aac63fa2faea6e3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp38-none-win_amd64.whl
| Download URL | bytewax-0.7.1-cp38-none-win_amd64.whl |
|---|---|
| Size | 617.7 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
255745a624cbbbb057a3f5d9fb6966faa57f6dd49b66fd09fc482943ee1c5a17
|
|
BLAKE2b-256 checksum How to use checksums |
c9109ddb4a7994184694a62c1556eb5026aae0a9d39e7797ded0fdc854917da1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
| Download URL | bytewax-0.7.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.8 Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
4fc40b5ca881ca74a0563f0c223dfa2714b5f84144af38544e01fa5df9a3d742
|
|
BLAKE2b-256 checksum How to use checksums |
df2bfabb9e1dda44f7cf105c95ca6aef59157986705bf956aadbfc7a196f64a7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
| Download URL | bytewax-0.7.1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
d771d21345e105115c8b5f2a2ab6c2a89e414ccf9b14db3c26eb5af072d22d8e
|
|
BLAKE2b-256 checksum How to use checksums |
6446e24c68cbf1fb11ce8375aef2c2cbfa9f29fc2d1e79135863046848f89e64
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp37-none-win_amd64.whl
| Download URL | bytewax-0.7.1-cp37-none-win_amd64.whl |
|---|---|
| Size | 617.9 kB |
| Tags | CPython 3.7 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
283341a4b4da7c4b4261526104ae43f8bc5b42b78ed3e7b45f4758757fd94ba0
|
|
BLAKE2b-256 checksum How to use checksums |
14600ec71555a2d721e2f724ebc26e3017009b1621337e4c23311c41edffa84e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
| Download URL | bytewax-0.7.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
58a7b4eebaf3ee318a37c93fd8ddc25c7c9d84096c9865109234d8ac7ed143f3
|
|
BLAKE2b-256 checksum How to use checksums |
ef165ff00247f64803604df1a560dc1c0eb5685ecaebb439066c3ab5a74e5097
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
Release files / bytewax-0.7.1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
| Download URL | bytewax-0.7.1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
01f05ffafd02b61e758dd2cfeace45fdd56e402e4bdb30b6d22f66a2cc74bd2f
|
|
BLAKE2b-256 checksum How to use checksums |
6fecc432147a7259b72cc2efdffd4b45a0fd35c82d27cc9692ab3ac7214f7ee4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.11.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.9.10
|
