This release is a pre-release and may not be stable for production use.
Kstreams
kstreams is a library/micro framework to use with kafka. It has simple kafka streams implementation that gives certain guarantees, see below.
Documentation: https://kpn.github.io/kstreams/
Installation
pip install kstreams
You will need a worker, we recommend aiorun
pip install aiorun
Usage
import aiorun
from kstreams import create_engine, ConsumerRecord
stream_engine = create_engine(title="my-stream-engine")
@stream_engine.stream("local--kstream")
async def consume(cr: ConsumerRecord):
print(f"Event consumed: headers: {cr.headers}, payload: {cr.value}")
async def produce():
payload = b'{"message": "Hello world!"}'
for i in range(5):
metadata = await stream_engine.send("local--kstreams", value=payload)
print(f"Message sent: {metadata}")
async def start():
await stream_engine.start()
await produce()
async def shutdown(loop):
await stream_engine.stop()
if __name__ == "__main__":
aiorun.run(start(), stop_on_unhandled_errors=True, shutdown_callback=shutdown)
Features
- Produce events
- Consumer events with
Streams - Subscribe to topics by
pattern -
Prometheusmetrics and custom monitoring - TestClient
- Custom Serialization and Deserialization
- Easy to integrate with any
asyncframework. No tied to any library!! - Yield events from streams
- Opentelemetry Instrumentation
- Middlewares
- Hooks (on_startup, on_stop, after_startup, after_stop)
- Store (kafka streams pattern)
- Stream Join
- Windowing
Development
This repo requires the use of poetry instead of pip.
Note: If you want to have the virtualenv in the same path as the project first you should run poetry config --local virtualenvs.in-project true
To install the dependencies just execute:
poetry install
Then you can activate the virtualenv with
poetry shell
Run test:
./scripts/test
Run code formatting with ruff:
./scripts/format
Commit messages
We use conventional commits for the commit message.
The use of commitizen is recommended. Commitizen is part of the dev dependencies.
cz commit
3 maintainers
Metadata
3 maintainers
Release files for kstreams 0.26.8rc0.dev143582937941
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| kstreams-0.26.8rc0.dev143582937941.tar.gz | 30.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| kstreams-0.26.8rc0.dev143582937941-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 69.2 kB
Release files / kstreams-0.26.8rc0.dev143582937941.tar.gz
| Download URL | kstreams-0.26.8rc0.dev143582937941.tar.gz |
|---|---|
| Size | 30.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dd5901c6bc3d0a3d93ffd388833abc9978239664c21941347ab69eacf7e110ea
|
|
BLAKE2b-256 checksum How to use checksums |
18641009a5abb796bf084799c6dd5fc52084e2c1c5c1947d4b4e87f00ebdae18
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.2 CPython/3.12.9 Linux/6.8.0-1021-azure
|
Release files / kstreams-0.26.8rc0.dev143582937941-py3-none-any.whl
| Download URL | kstreams-0.26.8rc0.dev143582937941-py3-none-any.whl |
|---|---|
| Size | 38.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0771e202ca37a043d1661cf1a31d624a80845cf97faad0c97a87b00a31669762
|
|
BLAKE2b-256 checksum How to use checksums |
cf858cec091188d7214bbc0dd1f16010f2f7b800be8e2ab4bcb695f05852673e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.2 CPython/3.12.9 Linux/6.8.0-1021-azure
|
