Python SDK for Numaflow
pynumaflow is the Python SDK for Numaflow, a Kubernetes-native stream processing framework. Write a Python function, wire it to a server class, and Numaflow handles the gRPC transport, autoscaling, and deployment — no boilerplate required. The SDK supports synchronous and asynchronous execution models, and both function-based and class-based handler styles.
Installation
pip install pynumaflow
Build & develop locally
This project uses uv for dependency management and packaging. To build the package locally, run the following command from the root of the project.
make setup
To run unit tests:
make test
To format code style using black and ruff:
make lint
Setup pre-commit hooks:
pre-commit install
Capabilities
The SDK covers the full range of Numaflow extension points. Each capability maps to a dedicated set of server classes and handler interfaces.
[!TIP] Each capability below links to working examples in both function-based and class-based handler styles. See the full examples directory for all implementations.
| Description | API Reference | |
|---|---|---|
| User-Defined Functions (UDFs) | Process and transform stream data — Map, Reduce, Reduce Stream, Map Stream, Batch Map, Accumulator | Map · Reduce · Reduce Stream · Map Stream · Batch Map · Accumulator |
| User-Defined Sources (UDSource) | Ingest data from custom sources with read, ack, pending, and partition handlers | Sourcer · Source Transform |
| User-Defined Sinks (UDSink) | Deliver data to custom destinations with per-message acknowledgment | Sinker |
| Side Inputs | Broadcast slow-changing reference data to UDF vertices without passing it through the pipeline | Side Input |
Choosing Your Server Type
Each functionality is served by a dedicated server class. Choose the server type that matches your workload characteristics:
| Sync | Async | |
|---|---|---|
| Concurrency Model | Multithreaded | asyncio event loop |
| Handler Signature | def handler(...) |
async def handler(...) |
| GIL Behaviour | Subject to GIL | Subject to GIL |
| Typical Workloads | Stateless transforms | I/O-bound operations |
Server Class Reference
All server types accept handlers in two styles:
- Function-based — pass a plain
deforasync defdirectly to the server. Best for simple, stateless logic. - Class-based — inherit from the corresponding base class (e.g.,
Mapper,Reducer,Sinker) and implement thehandlermethod. Useful when your handler needs initialization arguments, internal state, or helper methods.
The linked examples above demonstrate both styles for each functionality.
Contributing
For SDK development workflow, testing against a live pipeline, and adding new examples, see the Developer Guide. For general contribution guidelines, see the Numaproj Contributing Guide.
Release files for pynumaflow 0.14.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pynumaflow-0.14.0.tar.gz | 1.6 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pynumaflow-0.14.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.8 MB
Release files / pynumaflow-0.14.0.tar.gz
| Download URL | pynumaflow-0.14.0.tar.gz |
|---|---|
| Size | 1.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7a5db59227c65395a4ab80776b662ca4794f7188630b4c8773be75672714360a
|
|
BLAKE2b-256 checksum How to use checksums |
3806e50eae8c1228f515fe2f9a8bedddcc295bd60cc4fa8312b88f832bf653ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.26 {"installer":{"name":"uv","version":"0.11.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pynumaflow-0.14.0-py3-none-any.whl
| Download URL | pynumaflow-0.14.0-py3-none-any.whl |
|---|---|
| Size | 173.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ead3a59a93a6bfbcb9c7fb1fdb07254e568d8188012d08ae4b8bf8ade2098b43
|
|
BLAKE2b-256 checksum How to use checksums |
7649a4b00d138ce710c45bf6c651f8ad33753b4abcc530ecf69de2271df49670
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.26 {"installer":{"name":"uv","version":"0.11.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
