Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Push/Pull on Kafka over HTTP

Project Description

flasfka

Push/Pull on Kafka over HTTP

Installation

You can install kafka with pip:

pip install kafka

Alternatively, you can build and use the debian package.

Configuration

To set up flasfka to talk to the Kafka cluster, the following variables are available. Put this in a file (for example /etc/flasfka.conf.py):

# Hosts in the kafka cluster (list of ip:port)
HOSTS=["localhost:9092"]

# You probably don't need to change these

# How long to wait when polling a topic for new messages
CONSUMER_TIMEOUT=0.1

# Maximum of messages returned when consuming from a topic
CONSUMER_LIMIT=100

# When you don't specify a group for listening to a topic, flasfka
# will use this one
DEFAULT_GROUP="flasfka"

The file will be read at flasfka startup, provided it is in the environment variable FLASFKA_CONFIG:

export FLASFKA_CONFIG=/etc/flasfka.conf.py

Running flasfka is then just a matter of launching:

flasfka-serve

By default, it runs on the port 5000. Use flasfka-serve -h to learn how to change that. The debian package wraps flasfka into a uwsgi service, so you don’t need to worry about this if you are using it.

Usage

Now, assuming flasfka is configured and running:

send

curl -X POST --data-binary '{"messages": ["msg1", "msg2"]}' "http://127.0.0.1:5000/my-topic/"

This pushes msg1 then msg2 to the topic my-topic.

curl -X POST --data-binary '{"messages": ["msg3"]}' "http://127.0.0.1:5000/my-topic/my-key/"

This pushes msg3 to the topic my-topic with the key my-key.

fetch

curl http://127.0.0.1:5000/my-topic/

This retrieves a maximum of 100 of the last messages posted to my-topic, from the default group flasfka.

curl http://127.0.0.1:5000/my-topic/my-group/

This retrieves a maximum of 100 of the last messages posted to my-topic, from the group my-group.

curl http://127.0.0.1:5000/my-topic/?limit=20

This retrieves a maximum of 20 of the last messages posted to my-topic, from the default group flasfka.

Limitations

Flafska only accepts json data, so your messages need to be in utf-8. If you want to pass arbitrary data, it is recommended to encode them to a suitable format before (e.g. base64).

Release History

Release History

This version
History Node

1.1.6

History Node

1.1.5

History Node

1.1.4

History Node

1.1.3

History Node

1.1.2

History Node

1.1.1

History Node

1.1.0

History Node

1.0.2

History Node

1.0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
flasfka-1.1.6.tar.gz (5.2 kB) Copy SHA256 Checksum SHA256 Source Feb 27, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting