Kafka serialize python library
Kser is a bundle of python library whose purpose is to serialize tasks to be executed on Kafka consumers. It supports farious extensions:
Transport
librdkafka
You can choose to use the C bindings using confluent-kafka:
$ pip install kser[confluent]
kafka-python
You can choose the pure python library kafka-python:
$ pip install kser[pykafka]
http
There is also a light version using HTTP ideal to produce messages (not recommended for consumption)
$ pip install kser[http]
dummy
There is a dummy version for tests, no additional import required, just use / patch with the dummy module.
Other
encrypt data
It is possible to encrypt messages in the Kafka queue using libsodium.
$ pip install kser[crypto]
prometheus exporter
You can export metrics using the prometheus format:
$ pip install kser[prometheus]
A few set of environment variables allow to manage the exporter:
Environment variable |
Default value |
|---|---|
KSER_METRICS_ENABLED |
no |
KSER_METRICS_ADDRESS |
0.0.0.0 |
KSER_METRICS_PORT |
8888 |
The exporter has only 2 metrics defined by default, it’s just a sample. A good way to implement your own is to override the triggers methods (prefixed with ‘_’) like the following example:
from kser import KSER_METRICS_ENABLED
from prometheus_client import Counter
from kser.entry import Entrypoint
MY_METRIC = Counter('kser_my_metric', 'a usefull metric')
class MyEntrypoint(Entrypoint):
def _run(self):
if KSER_METRICS_ENABLED == "yes":
MY_METRIC.inc()
return self.run()See also:
- prometheus_client documentation
Documentation of the python Prometheus client
opentracing support
You can use opentracing with Jaeger to follow operation and task:
$ pip install kser[opentracing]
Configuration can be set using environment variable:
Environment variable |
Default value |
|---|---|
JAEGER_HOST |
localhost |
See also:
- Opentracing support with Jaeger
Kser documentation
Add-ons
Flask extension: Flask integration.
Requirements
Python 3.x
Documentations
Project: http://kser.readthedocs.io/
Libsodium: https://download.libsodium.org/doc/
confluent-kafka-python: http://docs.confluent.io/current/clients/confluent-kafka-python
kafka-python: http://kafka-python.readthedocs.io/en/master/
Other links
Project issues: https://github.com/cdumay/kser/issues
License
Licensed under BSD 3-Clause License or https://opensource.org/licenses/BSD-3-Clause.
1 maintainer
Release files for kser 0.8.27
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| kser-0.8.27.tar.gz | 15.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| kser-0.8.27-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 37.1 kB
Release files / kser-0.8.27.tar.gz
| Download URL | kser-0.8.27.tar.gz |
|---|---|
| Size | 15.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2deb7f93c3927528812c69a7509ac8f2234d183fa9293f0fd098727a0844b490
|
|
BLAKE2b-256 checksum How to use checksums |
0af738db4a9e3311bccbc59dd665b0f0862617d69427c1e29186a851fbf5d91f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.14
|
Release files / kser-0.8.27-py3-none-any.whl
| Download URL | kser-0.8.27-py3-none-any.whl |
|---|---|
| Size | 21.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
abfb6fbc18b6579ab3444756b435089506455b791c49a37e1f7a997e64aac18c
|
|
BLAKE2b-256 checksum How to use checksums |
12069b8ecc8f690ad1406733d75ff8d37b830d94a56a4c370038f7a3cde28fda
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.14
|
