kontiki-monitor

kontiki-monitor watches a Kontiki registry and publishes alert.normalized.

A service missing from the fleet, or short of active instances, opens expected_service_missing or insufficient_active_instances. An instance that registers, leaves, or changes state opens instance_registered, instance_unregistered, or instance_state_changed. The first sight of an exception fingerprint opens exception_recorded; a quiet period recovers it. Three failed sentinel heartbeats open sentinel_unreachable.

Framework options (kontiki.*) are documented with Kontiki. Restart the service after a YAML change.

kontiki-monitor

kontiki-monitor:
  category: kontiki.registry
  poll_interval_seconds: 30
  expected_services:
    my-api-service:
      min_active: 1

RPC list_open_alerts returns the open fleet, exception, and sentinel snapshots, sorted by alert_id. In memory only.

Lifecycle events map one-to-one onto alert.normalized. registry.exception.recorded is fingerprinted on (service_name, exception_type, message). The first sight opens alert_id exception:{service}:{short_hash} with attributes.resolution=open. Repeats while that alert is open stay quiet. After exception_recover_after_seconds the same alert_id recovers.

When publish raises AmqpDisconnectedError, that alert.normalized attempt is skipped.

Silences are runtime state under silences_path, added and cleared over RPC or HTTP. They are kept across restarts.

kontiki-monitor.expected_services

Map of Registry service name → spec.

A missing service emits expected_service_missing. Too few active instances emit insufficient_active_instances. Each edge is published once.

kontiki-monitor.sentinel

When the block is present, a task calls Registry get_services and, only if that call returns, POSTs an empty body to url with Authorization: Bearer {token}. The client timeout is 10 seconds. A 2xx status is success. Any other status, a connection error, or a timeout is a failed POST. A failed get_services does not POST and does not count as a failed POST.

Three consecutive failed POSTs publish sentinel_unreachable once (alert_id sentinel:unreachable, severity critical, attributes.resolution=open, attributes.sentinel_url set to url). Further failures stay quiet. A successful POST before the third failure clears the streak. The first success while the alert is open publishes the same alert_id at severity low, with attributes.resolution=recovered. The subscription catalog exposes criterion sentinel_url (eq, contains).

kontiki-monitor:
  sentinel:
    url: https://sentinel.example/watchdogs/prod/heartbeat
    token: "change-me"
    interval_seconds: 60

Run

RabbitMQ and the registry come from stack/common.services.yaml. config/default.yaml is the service file. config/embedded.yaml adds the demo fleet and HTTP silences on port 8091.

docker run --rm \
  -v "$PWD/stack:/stack:ro" \
  -v "$PWD/config:/config:ro" \
  ghcr.io/kontiki-org/kontiki-monitor:1.0.0 \
  --config /stack/common.services.yaml \
  --config /config/default.yaml \
  --config /config/embedded.yaml

From a checkout:

poetry run kontiki-monitor \
  --config stack/common.services.yaml \
  --config config/default.yaml \
  --config config/embedded.yaml