Skip to main content

No project description provided

Project description

Bytewax

Actions Status PyPI Bytewax User Guide

Bytewax is an open source Python framework for building highly scalable dataflows in a streaming or batch context.

Dataflow programming is a programming paradigm where program execution is conceptualized as data flowing through a series of operations or transformations.

At a high level, Bytewax provides a few major benefits:

  • The operators in Bytewax are largely “data-parallel”, meaning they can operate on independent parts of the data concurrently.
  • The ability to express higher-level control constructs, like iteration.
  • Bytewax allows you to develop and run your code locally, and then easily scale that code to multiple workers or processes without changes.

Bytewax uses PyO3 to provide Python bindings to the Timely Dataflow Rust library.

Visit our documentation site for full documentation

Usage

Install the latest release with pip:

pip install bytewax

Example

Here is an example of a simple dataflow program using Bytewax:

# pyexamples/simple.py
from bytewax import Executor

ec = Executor()
flow = ec.Dataflow(enumerate(range(10)))
flow.map(lambda x: x * x)
flow.inspect(print)


if __name__ == "__main__":
    ec.build_and_run()

Running the program:

python ./pyexamples/simple.py
0
1
4
9
16
25
36
49
81
64

For a more complete example, and documentation on the available operators, check out the User Guide.

License

Bytewax is licensed under the Apache-2.0 license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bytewax-0.7.0.tar.gz (585.4 kB view hashes)

Uploaded Source

Built Distributions

bytewax-0.7.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB view hashes)

Uploaded PyPy manylinux: glibc 2.12+ x86-64

bytewax-0.7.0-cp310-none-win_amd64.whl (638.1 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

bytewax-0.7.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.12+ x86-64

bytewax-0.7.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.6 MB view hashes)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bytewax-0.7.0-cp39-none-win_amd64.whl (638.4 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

bytewax-0.7.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.12+ x86-64

bytewax-0.7.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.6 MB view hashes)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bytewax-0.7.0-cp38-none-win_amd64.whl (640.8 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

bytewax-0.7.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

bytewax-0.7.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.6 MB view hashes)

Uploaded CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

bytewax-0.7.0-cp37-none-win_amd64.whl (640.9 kB view hashes)

Uploaded CPython 3.7 Windows x86-64

bytewax-0.7.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

bytewax-0.7.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.6 MB view hashes)

Uploaded CPython 3.7m macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page