Provides the interfaces of writing Python User Defined Functions and Sinks for NumaFlow.
Project description
Python SDK for Numaflow
This SDK provides the interface for writing UDFs and UDSinks in Python.
Implement a User Defined Function (UDF)
from pynumaflow.function import Messages, Message, Datum, UserDefinedFunctionServicer
def map_handler(key: str, datum: Datum) -> Messages:
val = datum.value
_ = datum.event_time
_ = datum.watermark
messages = Messages()
messages.append(Message.to_vtx(key, val))
return messages
if __name__ == "__main__":
grpc_server = UserDefinedFunctionServicer(map_handler)
grpc_server.start()
Sample Image (TODO)
Implement a User Defined Sink (UDSink)
from typing import List
from pynumaflow.sink import Datum, Responses, Response, UserDefinedSinkServicer
def udsink_handler(datums: List[Datum]) -> Responses:
responses = Responses()
for msg in datums:
print("User Defined Sink", msg)
responses.append(Response.as_success(msg.id))
return responses
if __name__ == "__main__":
grpc_server = UserDefinedSinkServicer(udsink_handler)
grpc_server.start()
Sample Image
A sample UDSink Dockerfile is provided under examples.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pynumaflow-0.2.3.tar.gz
(17.8 kB
view hashes)
Built Distribution
pynumaflow-0.2.3-py3-none-any.whl
(26.5 kB
view hashes)
Close
Hashes for pynumaflow-0.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 68943e8e5b1754fa84f0255aa378d15d10113f3a13f3dbef2b213fe746b5f3e0 |
|
MD5 | 7d3c1a23b429b9b29bdebd24a7bcdc4d |
|
BLAKE2b-256 | b01a04e4f7b50c2e04738bd1453dba38a733cdf56fe810c5b1a3c1fd2aae6e73 |