Kafka serialize python library
Kser is a bundle of python library whose purpose is to serialize tasks to be executed on Kafka consumers. It supports farious extensions:
Transport
librdkafka
You can choose to use the C bindings using confluent-kafka:
$ pip install kser[confluent]
kafka-python
You can choose the pure python library kafka-python:
$ pip install kser[pykafka]
http
There is also a light version using HTTP ideal to produce messages (not recommended for consumption)
$ pip install kser[http]
dummy
There is a dummy version for tests, no additional import required, just use / patch with the dummy module.
Other
encrypt data
It is possible to encrypt messages in the Kafka queue using libsodium.
$ pip install kser[crypto]
prometheus exporter
You can export metrics using the prometheus format:
$ pip install kser[prometheus]
A few set of environment variables allow to manage the exporter:
Environment variable |
Default value |
|---|---|
KSER_METRICS_ENABLED |
no |
KSER_METRICS_ADDRESS |
0.0.0.0 |
KSER_METRICS_PORT |
8888 |
The exporter has only 2 metrics defined by default, it’s just a sample. A good way to implement your own is to override the triggers methods (prefixed with ‘_’) like the following example:
from kser import KSER_METRICS_ENABLED
from prometheus_client import Counter
from kser.entry import Entrypoint
MY_METRIC = Counter('kser_my_metric', 'a usefull metric')
class MyEntrypoint(Entrypoint):
def _run(self):
if KSER_METRICS_ENABLED == "yes":
MY_METRIC.inc()
return self.run()See also:
- prometheus_client documentation
Documentation of the python Prometheus client
opentracing support
You can use opentracing with Jaeger to follow operation and task:
$ pip install kser[opentracing]
Configuration can be set using environment variable:
Environment variable |
Default value |
|---|---|
JAEGER_HOST |
localhost |
See also:
- Opentracing support with Jaeger
Kser documentation
Add-ons
Flask extension: Flask integration.
Requirements
Python 3.x
Documentations
Project: http://kser.readthedocs.io/
Libsodium: https://download.libsodium.org/doc/
confluent-kafka-python: http://docs.confluent.io/current/clients/confluent-kafka-python
kafka-python: http://kafka-python.readthedocs.io/en/master/
Other links
Project issues: https://github.com/cdumay/kser/issues
License
Licensed under BSD 3-Clause License or https://opensource.org/licenses/BSD-3-Clause.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file kser-0.8.27.tar.gz.
File metadata
- Download URL: kser-0.8.27.tar.gz
- Upload date:
- Size: 15.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.11.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2deb7f93c3927528812c69a7509ac8f2234d183fa9293f0fd098727a0844b490
|
|
| MD5 |
1f63cafca328ca0d297fdad19e5fa4c4
|
|
| BLAKE2b-256 |
0af738db4a9e3311bccbc59dd665b0f0862617d69427c1e29186a851fbf5d91f
|
File details
Details for the file kser-0.8.27-py3-none-any.whl.
File metadata
- Download URL: kser-0.8.27-py3-none-any.whl
- Upload date:
- Size: 21.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.11.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
abfb6fbc18b6579ab3444756b435089506455b791c49a37e1f7a997e64aac18c
|
|
| MD5 |
2825c8af11b99c7604bbacefba0ca1db
|
|
| BLAKE2b-256 |
12069b8ecc8f690ad1406733d75ff8d37b830d94a56a4c370038f7a3cde28fda
|
