Skip to main content

No project description provided

Project description

pyconfluent

pyconfluent brings most Confluent Kafka functionality to Python by providing wrappers for the KSQL and Schema Registry REST APIs, and an in-depth Pythonic interpretation of the Kafka Streams Java package inspired by Robinhood's faust and Winton's winton-kafka-streams.

Installation

This package was written for Python 3.6 and was not tested on other versions.

pip3 install pyconfluent

Usage

pyconfluent requires the Confluent Platform and all its underlying services to be running. When creating class instances, make sure to pass in the list of brokers running or leave it empty to connect to localhost.

KSQL

import KSQL

k = KSQL()  # enter your boostrap_server here if not 'localhost', no port

# create streams from existing topics
sales_stream = k.ksql("CREATE STREAM sales (company VARCHAR, product BIGINT, quantity BIGINT)"
                      "WITH (KAFKA_TOPIC='sales', VALUE_FORMAT='JSON';")

reviews_stream = k.ksql("CREATE STREAM reviews (company VARCHAR, product BIGINT, review VARCHAR)"
                        "WITH (KAFKA_TOPIC='sales', VALUE_FORMAT='JSON';")

# stream to stream join, WITHIN clause required
# creates and populates new kafka topic
enriched_stream = k.ksql("CREATE STREAM enriched AS SELECT"
                         "sales.company, sales.product, sales.quantity"
                         "FROM sales LEFT JOIN reviews WITH 1 HOURS"
                         "ON sales.company = reviews.company")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyconfluent, version 0.0.6
Filename, size File type Python version Upload date Hashes
Filename, size pyconfluent-0.0.6-py3-none-any.whl (45.2 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size pyconfluent-0.0.6.tar.gz (3.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page