A mini-framework for Kafka apps
Project description
Kaf
Så er det tid til en kop kaf'
Kaf is a small Python framework for creating Kafka apps. It is inspired by Faust, but differs in the following ways:
- Kaf is synchronous (async is future work)
- Kaf is compatible with Azure Eventhubs (over Kafka interface)
- Kaf is designed to work with different brokers for the consumer and producer
The framework depends on Confluent Kafka.
How to use
Hello world:
import logging
from kaf import KafkaApp
consumer_conf = {'bootstrap.servers': 'kafka:9092', 'group.id': 'myapp'}
producer_conf = {'bootstrap.servers': 'kafka:9092'}
app = KafkaApp('myapp', consumer_conf, producer_conf)
app.logger.setLevel(logging.INFO)
@app.process(topic='foo', publish_to='bar')
def uppercase_messages(msg):
return {'message_upper': msg['message'].upper()}
if __name__ == '__main__':
app.run()
How errors are handled
In case of an unhandled error, the framework will reinitialise the consumer and producer and wait for 3 seconds.
Future work:
Features to be added:
- Add decorators for app events
on_consume
,on_processed
andon_publised
. This will allow to hook up e.g. Datadog metrics to these events.
Tutorials followed while writing this code
Links:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kaf-0.1.1a0.tar.gz
(3.1 kB
view hashes)