fluvio 0.20.0

Python client library for Fluvio

Fluvio Client for Python

Python binding for Fluvio streaming platform.

Documentation

Fluvio client uses pdoc to generate the client API documentation.

Installation

pip install fluvio

This will get the wheel for the os/architecture of the installation system if available, otherwise it will try to build from source. If building from source, you will need the rust compiler and maybe some operating system sources.

Example Usage

Creating a Topic

fluvio_admin = FluvioAdmin.connect()
fluvio_admin.create_topic("a_topic")

Or just create a topic with custom settings:

import fluvio

fluvio_admin = FluvioAdmin.connect()
topic_spec = (
    TopicSpec.create()
    .with_retention_time("1h")
    .with_segment_size("10M")
    .build()
)
fluvio_admin.create_topic("a_topic", topic_spec)

Producer

from fluvio import Fluvio
fluvio = Fluvio.connect()
producer = fluvio.topic_producer('my-topic')
producer.send_string("FOOBAR")
producer.flush()

Consumer

from fluvio import (Fluvio, Offset)
fluvio = Fluvio.connect()
consumer = fluvio.partition_consumer('my-topic', 0)
stream = consumer.stream(Offset.beginning())

for i in stream:
    print(i.value_string())

Developer Notes

This project uses PyO3 to wrap the fluvio crate.

setuptools-rust bundles it into a python package. For cross platform builds, cibuildwheel is used.

Running the tests locally require having already setup a fluvio locally or on fluvio cloud.

Add python unit tests in the tests directory using the built in python unittest framework

You should probably stick to using make integration-tests which will create the virtual environment and install the package in the site-packages in the venv directory. This makes sure that the package is also packaged correctly.

If you'd like more rapid testing, once you've got the virtual environment activated, python setup.py test will compile the rust as a static library and put it as fluvio/fluvio_python.cpython-39-x86_64-linux-gnu.so. This filename is dependent on the host OS and python version. FLUVIO_CLOUD_TEST_PASSWORD` to your fork's secrets.

When submitting a PR, CI checks a few things:

• make integration-tests against a fluvio cluster in CI.
• make macos-ci-tests with no fluvio cluster present (the macOS github runner is flakey) to verify linking is done correctly.
• make lint. This checks that cargo fmt, flake8 and black are all clear.