kio 0.3.0

Python data types for the Apache Kafka® Protocol.

kio

Python data types for the Apache Kafka® Protocol.

Checkout the complete documentation →

Features

• Exposes immutable dataclass entities for all protocol messages, generated from the same source as used internally in Apache Kafka®.
• Message classes are simple light-weight data containers that do not inherit anything or expose any methods other than a regular dataclass. Encoding and decoding is enabled by making all the necessary details about Kafka encoding introspectable.
• Widely compatible encoding and decoding of messages through the IO[bytes] interface.
• Test suite with focus on roundtrip property tests using Hypothesis, including compatibility testing against the internals of upstream Apache Kafka®.

Installation

$ pip install --require-virtualenv kio

Development

Install development requirements.

$ pip install --require-virtualenv -e .[all]

The test suite contains integration tests that expects to be able to connect to an Apache Kafka® instance running on 127.0.0.1:9092. There is a Docker Compose file in container/compose.yml that you can use to conveniently start up an instance.

$ docker compose up -d kafka

Run tests.

$ python3 -X dev -m pytest --cov

Setup pre-commit to run on push.

$ pre-commit install -t pre-push

[!WARNING]
Building the schema will delete the src/kio/schema directory and recreate it again, hence all of the files under this directory will be deleted. Make sure to not put unrelated files there and accidentally wipe out your own work.

Fetch, generate, and format schema.

$ make build-schema