Skip to main content

A tool for monitoring tasks and connectors in Kafka Connect.

Project description

ConnectMon

A tool for monitoring tasks and connectors in Kafka Connect.

Installation

With pip

pip install connectmon

With poetry

poetry add connectmon

Usage

from connectmon import env, API
from connectmon.messaging import TeamsService


## Setup Kafka Connect Rest API client and check if cluster is reachable
connect = API(env.CONNECT_URL)

## Get all connectors and check if any are in a failed state
connectors = connect.get_all_connectors()

## Loop through all channels and send messages
for channel in env.CHANNELS.channels:
    if channel.type == "teams":
        service = TeamsService(connect, channel)
        service.process_channel_connectors(connectors)
        service.send_message()

Configuration

Base configuration for ConnectMon is handled through Environment Variables.

Environment Variables

Name Type Description Default
CONFIG_PATH Optional string The path to the channel configuration file ""
CONNECT_URL string The URL of the Connect cluster "http://localhost:8083"
SKIP_TLS_VERIFY boolean Whether to skip TLS verification False
ENVIRONMENT string The environment the application is running in "dev"
LOG_LEVEL string The log level for the application "INFO"
LOG_FORMAT string The log format for the application "(asctime)s - ..."

Name in bold are required!

If CONFIG_PATH is provided, the application will attempt to load configuration from the file, which will set settings.CHANNELS with the channels configured in the supplied configuration file.

Channel Configuration

You can configure specific channels to receive notifications when connectors or tasks are paused or failed.

A config file could look like this

channels:
  - name: my-teams-team-name
    type: teams
    url: https://my-org.webhook.office.com/webhookb2/...
    actions:
      - RESTART_FAILED
    include:
      - i-want-to-monitor-this-connector
      - this-too
    exclude:
      - who-cares-about-this-connector
      - this-is-someone-elses-problem

Supported fields for channels are:

Name Type Description Default
name string Name of the channel ""
type string Type of channel ""
url string Url to send payload to ""
actions list of strings Can be any of RESTART_FAILED, RESTART_FAILED_CONNECTORS, RESTART_FAILED_TASKS, RESUME_PAUSED_CONNECTORS RESTART_FAILED
include list of strings Names of connectors to include for this channel ["*"]
exclude list of strings Names of connectors to exclude for this channel []

Current only Microsoft Teams or type: "teams" is supported.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

connectmon-0.2.7.tar.gz (8.1 kB view details)

Uploaded Source

Built Distribution

connectmon-0.2.7-py3-none-any.whl (9.3 kB view details)

Uploaded Python 3

File details

Details for the file connectmon-0.2.7.tar.gz.

File metadata

  • Download URL: connectmon-0.2.7.tar.gz
  • Upload date:
  • Size: 8.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.2.2 CPython/3.8.10 Darwin/22.3.0

File hashes

Hashes for connectmon-0.2.7.tar.gz
Algorithm Hash digest
SHA256 6876ec94251ad46925176b76425e72f89cd8f0fd727d24448bbb0c44b2c783b3
MD5 ec6e6ded4c177c76eb83bb2324948a0a
BLAKE2b-256 bf6b10ac380b50871f1c17b42f9e7577498e6947f2d42be6330df69868fcb93a

See more details on using hashes here.

Provenance

File details

Details for the file connectmon-0.2.7-py3-none-any.whl.

File metadata

  • Download URL: connectmon-0.2.7-py3-none-any.whl
  • Upload date:
  • Size: 9.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.2.2 CPython/3.8.10 Darwin/22.3.0

File hashes

Hashes for connectmon-0.2.7-py3-none-any.whl
Algorithm Hash digest
SHA256 7f2bec95ec552ad756d37d0e7b52934b580ad86a90aa8cd39e4d71d3072e4ab1
MD5 6d34bbeff93a14c7cce067ba6c50905e
BLAKE2b-256 d1e45e8ac5fde7fb6ba781a9cc1df07b44160d63d3edc3d0f833c9364eec4729

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page