Skip to main content

Description

This package contains the Kafka implementation of the Input-/OutputPortPlugin interface of pypz. It enables the operators to send and receive data records in a real streaming fashion.

Check the documentation for more details.

Install

The python artifact is hosted on https://pypi.org/, so you can install it via pip:

pip install pypz-kafka-io

If you want to work on it locally, then you should install in editable mode:

pip install -e ./plugins/kafka_io

Test

You can run tests against an existing Kafka cluster, however, it is more convenient and safe to use a local test cluster.

  1. Run the confluent image to start a local cluster
docker run --name kafka-test-cluster -d --rm -p 9092:9092 confluentinc/confluent-local
  1. Install the subproject in editable more.
pip install -e ./plugins/kafka_io
  1. To run the tests locally, you need to execute the following command:
python -m unittest discover .\plugins\kafka_io\test\ -p "*.py"
  1. Delete cluster
docker stop kafka-test-cluster

Known issues

Due to the fact that the test execution is sometimes faster than the Kafka cluster, there might be failed tests. Although there are already several delays inserted into the test code, it still can happen. Try to rerun the test and give a notification to the maintainer(s). Thank you.

Build

Before you build, you will need to install the "build" package:

pip install build

Optionally, you might want to set a version for local testing. Using setuptools-scm, you can pretend a version via setting the following environment variable:

# Powershell
$env:SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYPZ_KAFKA_IO="v1.2.3"

# Linux
export SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYPZ_KAFKA_IO="v1.2.3"

Then from the subproject's root, you will need to execute the following command:

python -m build

It will create the source distribution and the wheel file in the "dist" folder.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pypz_kafka_io-0.10.2.tar.gz (19.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pypz_kafka_io-0.10.2-py3-none-any.whl (13.9 kB view details)

Uploaded Python 3

File details

Details for the file pypz_kafka_io-0.10.2.tar.gz.

File metadata

  • Download URL: pypz_kafka_io-0.10.2.tar.gz
  • Upload date:
  • Size: 19.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for pypz_kafka_io-0.10.2.tar.gz
Algorithm Hash digest
SHA256 8b044f1f09af20b83fa3a96c8ae4c7ef26409d9e52a2ca9440b5371d757eb9c1
MD5 ab4e533aedf1efb966310eebe0ed0ee1
BLAKE2b-256 8f43c20b17a12aa2088ef6778f4ac81c54c7b096ef60303ce5c34d220597555c

See more details on using hashes here.

Provenance

The following attestation bundles were made for pypz_kafka_io-0.10.2.tar.gz:

Publisher: publish.yml on lazlowa/pypz-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pypz_kafka_io-0.10.2-py3-none-any.whl.

File metadata

  • Download URL: pypz_kafka_io-0.10.2-py3-none-any.whl
  • Upload date:
  • Size: 13.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for pypz_kafka_io-0.10.2-py3-none-any.whl
Algorithm Hash digest
SHA256 57391a6e0930373d5fa48df0b6ad6cd9bc85fc297eb9216ea54f564b4e920b19
MD5 009b727121896e1bb42cab1b944de59d
BLAKE2b-256 3b389d0eac1f8994673d74a207ecf76e924a5318eede158ed0379055f1b93349

See more details on using hashes here.

Provenance

The following attestation bundles were made for pypz_kafka_io-0.10.2-py3-none-any.whl:

Publisher: publish.yml on lazlowa/pypz-python

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.10.2 This release

2 files

0.10.1

2 files

0.10.0

2 files

0.9.6

2 files

0.9.5

2 files

0.9.4

2 files

0.9.3

2 files

0.9.2

2 files

0.9.1

2 files

0.9.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page