kuhl-haus-magpie

Site monitoring and synthetic transaction service built on Django and Celery.

Magpie runs scheduled canary checks against your HTTP endpoints and reports results to Graphite for dashboarding and alerting. It provides a Django Admin interface for managing endpoints and a REST API for programmatic access.

What It Does

Magpie performs three types of canary checks on configured endpoints:

Check	What It Does
HTTP Health Check	Sends a GET request and validates the response status code or JSON body
TLS Certificate Check	Connects via SSL and calculates days until certificate expiration
DNS Check	Queries configured DNS resolvers and validates the response

Results — including response times, error counts, and check-specific attributes — are posted to a Graphite server via the Carbon pickle protocol.

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Celery Beat  │────>│ Celery Worker│────>│  Canary Task │
│ (scheduler)  │     │              │     │  Functions   │
└──────────────┘     └──────────────┘     └──────┬───────┘
                                                 │
                           ┌─────────────────────┼─────────────────────┐
                           │                     │                     │
                     ┌─────▼───────┐       ┌─────▼──────┐      ┌───────▼─────┐
                     │ HTTP Health │       │  TLS Check │      │  DNS Check  │
                     │   Check     │       │            │      │             │
                     └─────┬───────┘       └─────┬──────┘      └───────┬─────┘
                           │                     │                     │
                           └─────────────────────┼─────────────────────┘
                                                 │
                                          ┌──────▼──────┐     ┌──────────┐
                                          │ CarbonPoster│────>│ Graphite │
                                          │  (pickle)   │     │  Server  │
                                          └─────────────┘     └──────────┘

┌──────────────┐     ┌──────────────┐
│ Django Admin │────>│  PostgreSQL  │  (SQLite for development)
│  (Unfold)    │     │   Database   │
└──────────────┘     └──────────────┘

┌──────────────┐
│  Django Web  │──── /health, /healthz, /admin/, /api/
│  (Gunicorn)  │
└──────────────┘

Celery Beat triggers canary tasks on a schedule you define through the Django Admin. Celery Workers execute the checks and batch-post metrics to your Graphite server. The Django web application provides endpoint management, health check endpoints, and a Swagger/ReDoc API.

Tech Stack

Component	Technology
Language	Python ≥ 3.12
Web framework	Django 5.2
Task queue	Celery + django-celery-beat + django-celery-results
API	Django REST Framework + drf-yasg (Swagger/ReDoc)
Admin theme	django-unfold
Metrics transport	Graphite Carbon pickle protocol
DNS checks	dnspython
TLS checks	pyOpenSSL
Database	PostgreSQL (production) / SQLite (development)
ASGI server	Uvicorn + Gunicorn

Quick Start

Install from PyPI

pip install kuhl-haus-magpie

Run with Docker

Pre-built container images are published to GitHub Container Registry. A full deployment requires four containers:

Image	Purpose
ghcr.io/kuhl-haus/kuhl-haus-magpie:latest	Django web application
ghcr.io/kuhl-haus/kuhl-haus-magpie-worker:latest	Celery worker (executes canary tasks)
ghcr.io/kuhl-haus/kuhl-haus-magpie-beat:latest	Celery Beat (task scheduler)
ghcr.io/kuhl-haus/kuhl-haus-magpie-flower:latest	Flower (Celery monitoring dashboard)

Each container runs docker-entrypoint.sh on startup, which bootstraps the database (migrations, static files, superuser creation).

Development Setup

git clone https://github.com/kuhl-haus/kuhl-haus-magpie.git
cd kuhl-haus-magpie
pip install -r requirements.txt
pdm install

# Run the Django development server
python -m kuhl_haus.magpie.manage runserver 0.0.0.0:8000

# Run tests
pdm run pytest tests -v

# Run tests with coverage
pdm run pytest --cov=kuhl_haus.magpie --cov-report=html tests -v

Configuration

Magpie is configured entirely through environment variables. No configuration files are required.

Application Settings

Variable	Purpose	Default
DJANGO_SECRET_KEY	Django secret key	Insecure dev default
DJANGO_DEBUG	Enable debug mode	True
DJANGO_ALLOWED_HOSTS	Comma-separated allowed hosts	Derived from MAGPIE_DOMAIN
MAGPIE_DOMAIN	Domain for cookies, CSRF, and CORS	localhost
DISABLE_HTTPS	Disable HTTPS cookie/session settings	False

API Authentication

Variable	Purpose	Default
MAGPIE_UNSAFE_SETTING_DISABLE_API_AUTH	Restore the old, fully-open REST API behavior (no auth required for reads or writes). Deliberately named “unsafe” – the default requires a token or session login for any write (POST/PUT/PATCH/DELETE) to /api/endpoints/, /api/resolvers/, /api/resolver-lists/, /api/scripts/. Reads stay open either way.	False

By default, the REST API requires authentication for write requests. Obtain a token via the Django Admin (Auth Token section, once logged in as a superuser) or manage.py drf_create_token <username>, then send it as Authorization: Token <token>. The Swagger/ReDoc docs at /api/ and /redoc/ remain open regardless of this setting – that’s API documentation metadata, not data access.

This adds a new database table (rest_framework.authtoken, needed to store tokens). The packaged Docker image’s docker-entrypoint.sh runs the bootstrap management command on every start, which already calls migrate – no extra step needed there. If you run Magpie a different way (bare manage.py, a custom entrypoint/command override), run manage.py migrate yourself before creating or using any token; until then, token auth will fail even though the setting is on.

Session-authenticated (browser) callers are subject to Django’s normal CSRF protection on unsafe methods – if you’re driving the API from a browser session rather than a token, include the CSRF token/header or you’ll see an unexpected 403 that isn’t the auth check itself.

Database

Variable	Purpose	Default
POSTGRES_HOST	PostgreSQL host (if unset, uses SQLite)	None
POSTGRES_DB	Database name	magpie
POSTGRES_USER	Database user	magpie
POSTGRES_PASSWORD	Database password	magpie
POSTGRES_PORT	Database port	5432

Celery

Variable	Purpose	Default
CELERY_BROKER_URL	Message broker URL (Redis or RabbitMQ)	None

Metrics (Graphite)

Variable	Purpose	Default
CARBON_SERVER_IP	Graphite Carbon server IP	None
CARBON_PICKLE_PORT	Carbon pickle port	2004
NAMESPACE_ROOT	Root namespace for metrics	default
METRIC_NAMESPACE	Metric namespace	dev

Bootstrap

Variable	Purpose	Default
DJANGO_SUPERUSER_USERNAME	Initial admin username	admin
DJANGO_SUPERUSER_EMAIL	Initial admin email	admin@example.com
DJANGO_SUPERUSER_PASSWORD	Initial admin password (required for creation)	None

Container Metadata

Variable	Purpose	Default
IMAGE_VERSION	Image version (shown in /healthz)	Unknown
CONTAINER_IMAGE	Container image name (shown in /healthz)	Unknown
FLOWER_DOMAIN	Flower dashboard domain (shown on index page)	localhost:5555

Endpoints

Path	Method	Description
/	GET	Site map with links to admin, API docs, and Flower
/health	GET	Simple health check — returns OK (plain text)
/healthz	GET	Detailed health check — returns JSON with version info
/admin/	GET	Django Admin (Unfold theme)
/api/endpoints/	CRUD	Endpoint management API
/api/resolvers/	CRUD	DNS resolver management API
/api/resolver-lists/	CRUD	DNS resolver list management API
/api/scripts/	CRUD	Script configuration management API
/api/	GET	Swagger UI
/redoc/	GET	ReDoc API documentation

Metrics Format

Magpie posts metrics to Graphite in the following path structure:

{namespace}.{app_name}.mnemonic.{endpoint_mnemonic}.{metric_name}
{namespace}.{app_name}.hostname.{hostname}.{metric_name}

Tagged metrics (when metadata is configured):

{namespace}.{app_name}.mnemonic.{endpoint_mnemonic}.{metric_name};tag1=val1;tag2=val2

Counters

Every canary check records these counters:

Counter	Meaning
requests	Number of check attempts
responses	Successful responses
errors	Expected errors (bad status code, unhealthy response)
exceptions	Unexpected exceptions (timeouts, connection failures)

Check-Specific Attributes

HTTP Health Check: response_time, response_time_ms, status_code, text, request_time, request_time_ms, version

TLS Check: response_time, response_time_ms, days_until_expiration, expires_today, is_valid

DNS Check: response_time, response_time_ms, truncated, rcode, result

Managing Endpoints

Endpoints are managed through the Django Admin at /admin/. Each endpoint can be configured with:

Mnemonic — unique identifier used in metric paths
Hostname and port — target server
Environment — used for metric namespace grouping (e.g., prod, staging, dev)
Health check settings — scheme, path, expected status code, JSON response validation
TLS check — enable/disable certificate monitoring
DNS check — enable/disable with assigned resolver list
Timeouts — configurable connect and read timeouts
Ignore flag — temporarily disable an endpoint without deleting it

Canary task schedules are configured through Celery Beat in the Django Admin under Periodic Tasks.

Security

See SECURITY.md for vulnerability reporting guidelines
All releases are signed with Sigstore
CodeQL scanning runs on every push
Dependabot monitors pip and Docker dependencies

Contributing

See CONTRIBUTING.rst for development guidelines.

License

This project is licensed under the terms of the LICENSE.txt file.

Note

This project was set up using PyScaffold 4.6.

Metadata

Release files for kuhl-haus-magpie 0.6.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kuhl-haus-magpie 0.6.1
File	Size	Uploaded
kuhl_haus_magpie-0.6.1.tar.gz	55.7 kB	Sep 28, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for kuhl-haus-magpie 0.6.1
File	Interpreter	ABI	Platform	Reset
kuhl_haus_magpie-0.6.1-py3-none-any.whl 43.5 kB Sep 28, 2026	Python 3	none	any	Details

Total release size: 99.2 kB

Release files / kuhl_haus_magpie-0.6.1.tar.gz

Download URL	kuhl_haus_magpie-0.6.1.tar.gz
Size	55.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`1abd4b810fd9431071a64f41221065ae62d9b42aeda242d31ac2f4fdaa0f8e64`
BLAKE2b-256 checksum How to use checksums	`718676edc5f9ef74a0333a651ac4e9885d2fa1b02ce7206b8bc00842f54f51a7`
Upload date	Sep 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/kuhl-haus/kuhl-haus-magpie
Publishing commit: github.com/kuhl-haus/kuhl-haus-magpie/tree/3f832372a8a2f7282a63b297771523d8608aeead

Workflow

Publishing configuration: github.com/kuhl-haus/kuhl-haus-magpie/blob/3f832372a8a2f7282a63b297771523d8608aeead/.github/workflows/publish-to-pypi.yml
Publishing logs: github.com/kuhl-haus/kuhl-haus-magpie/actions/runs/36382384356/attempts/1

Artifact

Subject: kuhl_haus_magpie-0.6.1.tar.gz
SHA-256 checksum: 1abd4b810fd9431071a64f41221065ae62d9b42aeda242d31ac2f4fdaa0f8e64 Verifying attestations

Release files / kuhl_haus_magpie-0.6.1-py3-none-any.whl

Download URL	kuhl_haus_magpie-0.6.1-py3-none-any.whl
Size	43.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`f128018cfe30480ae332ea2abb30ecc8e46e824f9a079a28e4a25fed01751812`
BLAKE2b-256 checksum How to use checksums	`2a101abb6199d39ed56a140a0142dd43293e2144fc94b107d0aa651a385ac0ea`
Upload date	Sep 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/kuhl-haus/kuhl-haus-magpie
Publishing commit: github.com/kuhl-haus/kuhl-haus-magpie/tree/3f832372a8a2f7282a63b297771523d8608aeead

Workflow

Publishing configuration: github.com/kuhl-haus/kuhl-haus-magpie/blob/3f832372a8a2f7282a63b297771523d8608aeead/.github/workflows/publish-to-pypi.yml
Publishing logs: github.com/kuhl-haus/kuhl-haus-magpie/actions/runs/36382384356/attempts/1

Artifact

Subject: kuhl_haus_magpie-0.6.1-py3-none-any.whl
SHA-256 checksum: f128018cfe30480ae332ea2abb30ecc8e46e824f9a079a28e4a25fed01751812 Verifying attestations

Release history Release notifications | RSS feed

This release

0.6.1 This release

Sep 28, 2026 2 release files

0.6.0

Sep 12, 2026 2 release files

0.5.11

Jun 27, 2026 2 release files

0.5.9

May 4, 2026 2 release files

0.5.8

Mar 11, 2026 2 release files

0.5.7

Mar 11, 2026 2 release files

0.5.6

Mar 11, 2026 2 release files

0.5.5

Mar 11, 2026 2 release files

0.5.4

Mar 11, 2026 2 release files

0.5.2

Jan 23, 2026 2 release files

0.5.1

Jan 1, 2026 2 release files

0.5.0

Dec 28, 2025 2 release files

0.4.0

Jul 7, 2025 2 release files

0.3.2

Jul 5, 2025 2 release files

0.3.1

Jun 20, 2025 2 release files

0.2.5

May 22, 2025 2 release files

0.2.4

May 22, 2025 2 release files

0.2.2

May 22, 2025 2 release files

0.2.1

May 20, 2025 2 release files

0.2.0

May 20, 2025 2 release files

0.1.18

May 20, 2025 2 release files

0.1.17

May 20, 2025 2 release files

0.1.16

May 20, 2025 2 release files

0.1.15

May 20, 2025 2 release files

0.1.14

May 20, 2025 2 release files

0.1.13

May 20, 2025 2 release files

0.1.11

May 19, 2025 2 release files

0.1.10

May 9, 2025 2 release files

0.1.9

May 9, 2025 2 release files

0.1.8

May 8, 2025 2 release files

0.1.7

May 8, 2025 2 release files

0.1.6

May 7, 2025 2 release files

0.1.5

May 7, 2025 2 release files

0.1.4

May 5, 2025 2 release files

0.1.3

May 3, 2025 2 release files

kuhl-haus-magpie 0.6.1

kuhl-haus-magpie

What It Does

Architecture

Tech Stack

Quick Start

Install from PyPI

Run with Docker

Development Setup

Configuration

Application Settings

API Authentication

Database

Celery

Metrics (Graphite)

Bootstrap

Container Metadata

Endpoints

Metrics Format

Counters

Check-Specific Attributes

Managing Endpoints

Security

Contributing

License

Note

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License

Requires

Provides Extra

Classifiers

Metadata

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License

Requires

Provides Extra

Classifiers

Release files for kuhl-haus-magpie 0.6.1

Source distribution (sdist)

Built distribution (wheel)

Release files / kuhl_haus_magpie-0.6.1.tar.gz

Provenance

PyPI Publish Attestation

Identity

Workflow

Artifact

Release files / kuhl_haus_magpie-0.6.1-py3-none-any.whl

Provenance

PyPI Publish Attestation

Identity

Workflow

Artifact

Release history Release notifications | RSS feed