Skip to main content

A mini-framework for Kafka apps

Project description


Så er det tid til en kop kaf'

Kaf is a small Python framework for creating Kafka apps. It is inspired by Faust, but differs in the following ways:

  • Kaf is synchronous (async is future work)
  • Kaf is compatible with Azure Eventhubs (over Kafka interface)
  • Kaf is designed to work with different brokers for the consumer and producer

The framework depends on Confluent Kafka.

How to use

Hello world:

import logging

from kaf import KafkaApp

consumer_conf = {'bootstrap.servers': 'kafka:9092', '': 'myapp'}
producer_conf = {'bootstrap.servers': 'kafka:9092'}

app = KafkaApp('myapp', consumer_conf, producer_conf)


@app.process(topic='foo', publish_to='bar')
def uppercase_messages(msg):
  return {'message_upper': msg['message'].upper()}

if __name__ == '__main__':

How errors are handled

In case of an unhandled error, the framework will reinitialise the consumer and producer and wait for 3 seconds.

Future work:

Features to be added:

  • Add decorators for app events on_consume, on_processed and on_publised. This will allow to hook up e.g. Datadog metrics to these events.

How to deploy a new version

Steps (can maybe be improved):

  1. change version in
  2. git add + commit + push
  3. create new release in GitHub (copy link)
  4. update download_url in
  5. git add + commit + push (again)
  6. Run python sdist
  7. Run twine upload dist/* --verbose (if not installed, pip install twine first)

Useful links used:

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for kaf, version 0.1.2a0
Filename, size File type Python version Upload date Hashes
Filename, size kaf-0.1.2a0.tar.gz (3.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page