Skip to main content

Kafka and schema registry integration

Project description

kafka-schema-registry

This library allows you to create topics on Kafka topics, associated with a Confluent Schema Registry, and publish messages on them.

It takes care of:

  • creating the topic
  • publishing the associated schema (or updating an existing one)
  • serializing and publishing messages to Kafka

It works with kafka-python, and extra arguments are forwarded to it.

Installing

pip install kafka-schema-registry

Usage

from kafka_schema_registry import prepare_producer

SAMPLE_SCHEMA = {
    "type": "record",
    "name": "TestType",
    "fields" : [
        {"name": "age", "type": "int"},
        {"name": "name", "type": ["null", "string"]}
    ]
}


producer = prepare_producer(
        ['localhost:9092'],
        f'http://schemaregistry',
        topic_name,
        1,
        1,
        value_schema=SAMPLE_SCHEMA,
)

producer.send(topic_name, {'age': 34})
producer.send(topic_name, {'age': 9000, 'name': 'john'})

Running the tests

The test requires Docker in order to start a local Redpanda instance.

  • make start-redpanda to start the server
  • make test to configure a virtualenv and run the tests

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kafka-schema-registry-0.1.2.tar.gz (5.4 kB view hashes)

Uploaded Source

Built Distribution

kafka_schema_registry-0.1.2-py3-none-any.whl (5.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page