Skip to main content

Python SDK for Numaflow

Build black License Release Version

pynumaflow is the Python SDK for Numaflow, a Kubernetes-native stream processing framework. Write a Python function, wire it to a server class, and Numaflow handles the gRPC transport, autoscaling, and deployment — no boilerplate required. The SDK supports synchronous and asynchronous execution models, and both function-based and class-based handler styles.

Installation

pip install pynumaflow
Build & develop locally

This project uses uv for dependency management and packaging. To build the package locally, run the following command from the root of the project.

make setup

To run unit tests:

make test

To format code style using black and ruff:

make lint

Setup pre-commit hooks:

pre-commit install

Capabilities

The SDK covers the full range of Numaflow extension points. Each capability maps to a dedicated set of server classes and handler interfaces.

Description API Reference
User-Defined Functions (UDFs) Process and transform stream data — Map, Reduce, Reduce Stream, Map Stream, Batch Map, Accumulator Map · Reduce · Reduce Stream · Map Stream · Batch Map · Accumulator
User-Defined Sources (UDSource) Ingest data from custom sources with read, ack, pending, and partition handlers Sourcer · Source Transform
User-Defined Sinks (UDSink) Deliver data to custom destinations with per-message acknowledgment Sinker
Side Inputs Broadcast slow-changing reference data to UDF vertices without passing it through the pipeline Side Input

Choosing Your Server Type

Each functionality is served by a dedicated server class. Choose the server type that matches your workload characteristics:

Sync Async
Concurrency Model Multithreaded asyncio event loop
Handler Signature def handler(...) async def handler(...)
GIL Behaviour Subject to GIL Subject to GIL
Typical Workloads Stateless transforms I/O-bound operations

Server Class Reference

Functionality Server Class(es)
UDSource SourceAsyncServer
UDSink SinkServer, SinkAsyncServer
Side Input SideInputServer
Map MapServer, MapAsyncServer
Reduce ReduceAsyncServer
Reduce Stream ReduceStreamAsyncServer
Map Stream MapStreamAsyncServer
Batch Map BatchMapAsyncServer
Accumulator AccumulatorAsyncServer
Source Transform SourceTransformServer, SourceTransformAsyncServer

All server types accept handlers in two styles:

  • Function-based — pass a plain def or async def directly to the server. Best for simple, stateless logic.
  • Class-based — inherit from the corresponding base class (e.g., Mapper, Reducer, Sinker) and implement the handler method. Useful when your handler needs initialization arguments, internal state, or helper methods.

The linked examples above demonstrate both styles for each functionality.

Contributing

For SDK development workflow, testing against a live pipeline, and adding new examples, see the Developer Guide. For general contribution guidelines, see the Numaproj Contributing Guide.

Metadata

Release files for pynumaflow 0.15.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pynumaflow 0.15.0
File Size Uploaded Provenance Actions
pynumaflow-0.15.0.tar.gz 1.7 MB Without provenance Details for pynumaflow-0.15.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pynumaflow 0.15.0
File Interpreter ABI Platform Actions
pynumaflow-0.15.0-py3-none-any.whl Python 3 none any Details for pynumaflow-0.15.0-py3-none-any.whl

Total release size: 1.9 MB

Release files / pynumaflow-0.15.0.tar.gz

Download URL pynumaflow-0.15.0.tar.gz
Size 1.7 MB
Tags Source
SHA-256 checksum
How to use checksums
ab2ec98ab6ef195a701cbd388ec6cce8cb164a615cb8da44d92d0e50d4638d0c
BLAKE2b-256 checksum
How to use checksums
240e9bcc1402664c2b97989b036905227bd94b5a3964701e8afc085625fd708c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / pynumaflow-0.15.0-py3-none-any.whl

Download URL pynumaflow-0.15.0-py3-none-any.whl
Size 179.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
da00dd87a890ed515480ad561298ff2068186ba52a238b975f7f54afc243b55d
BLAKE2b-256 checksum
How to use checksums
9be6afcac11c4a360226be4c2c26c040bf571a2a5ee9bfab1cf0b12ffa14c290
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for pynumaflow 0.15.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.15.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page