alloy-config-generator 0.1.7

Deterministic Grafana Alloy configuration generator.

Alloy Config Generator

Deterministic configuration generator for Grafana Alloy.

Quick Start

Copy example definitions (public-safe) and generate:

# macOS / Linux
cp -r definitions.example definitions

# Windows (Command Prompt)
xcopy /E /I definitions.example definitions

uv tool run --from alloy-config-generator alloygen --all --format both

Optional: install globally with uv (requires PATH setup):

uv tool install alloy-config-generator
alloygen --all

If you want to run from GitHub directly:

uv tool run --from "git+ssh://[e-mail]/jski/alloy-config-generator.git@main" alloygen --all

If uv isn’t found, add the directory that contains uv.exe to your PATH. On Windows, run this in Command Prompt:

where uv

If alloygen still isn’t found after uv tool install, use uv tool list --verbose to locate alloygen.exe and add that folder to PATH (or stick with uv tool run --from ...).

Local (repo) usage:

python generate.py --all

Generate example outputs (deterministic, committed to repo):

alloygen --examples

Structure

definitions.example/ # public-safe example inputs
definitions/         # private, real configs (ignored by git)
  host-label-policy.yaml # optional centralized host label policy
  endpoints/         # where data is sent
  scrapes/           # what is collected
  hosts/             # which host uses what
  stacks/            # reusable scrape groups

templates/           # optional template overrides
generated/           # output (ignored by git)
  alloy/             # alloy configs (when using --format all/argocd)
  k8s/               # k8s manifests (when using --format all/argocd)

Templates are packaged with the tool. Create a local templates/ folder only if you want to override them.

Public / Private Repo Pattern

• Public repo ships definitions.example/ with placeholder values.
• Private repo stores real definitions/ (ignored in public).

Quick start for a new repo:

cp -r definitions.example definitions

You can also point to a different definitions directory:

alloygen --definitions-dir /path/to/definitions --all

Or set it once:

setx ALLOYGEN_DEFINITIONS_DIR C:\path\to\definitions

Core Concepts

• Endpoint: a Loki/Prometheus destination
• Scrape: logs or metrics definition
• Host: deployment target (docker or kubernetes)
• Stack: reusable set of scrapes

Minimal Example

Endpoint (definitions/endpoints/local.yaml):

name: local
prometheus:
  enabled: true
  url: https://prom.example.com/api/v1/write
  auth_type: basic
  username: admin
  password: ${PROMETHEUS_PASSWORD}

Scrape (definitions/scrapes/node.yaml):

name: node
type: metrics
endpoint: 127.0.0.1:9100
scrape_interval: 30s
metrics_path: /metrics
labels:
  job: node

For type: metrics, preferred input is host:port with explicit metrics_path. Full URL inputs are also supported and normalized automatically:

• __address__ uses host:port
• metrics_path is derived from the URL path

Host (definitions/hosts/host-01.yaml):

name: host-01
deployment_type: docker
endpoint: local
scrapes:
  - node
extra_labels:
  site: lan
configmap_name: alloy-config-host-01

Generate:

alloygen host-01

Common Patterns

Redundant routing (multiple sources of truth):

endpoints:
  prometheus: [optimus-grafana, bumblebee-grafana]
  loki: [optimus-grafana, bumblebee-grafana]

Multi-target scrape:

targets:
  - name: node_a
    address: node-a:9100
  - name: node_b
    address: node-b:9100

Syslog listener scrape:

name: unifi-activity-logs
type: logs-syslog
listener_address: 0.0.0.0:5514
protocol: udp
syslog_format: raw
listener_labels:
  listener: unifi-activity-logging
labels:
  job: unifi-activity-logs
  component: unifi

Centralized host labels (optional definitions/host-label-policy.yaml):

defaults:
  docker:
    platform: docker
  kubernetes:
    platform: kubernetes

identity_label_by_deployment:
  docker: host
  kubernetes: cluster

host_overrides:
  shockwave:
    env: prod

Label precedence is:

1. defaults[deployment_type]
2. host_overrides[host.name]
3. identity label (host/cluster) set to host name
4. host extra_labels (last-write-wins)

You can also switch metrics label injection mode:

alloygen --all --label-policy-mode static   # default
alloygen --all --label-policy-mode relabel  # shared host policy relabel component

The committed definitions.example/host-label-policy.yaml shows a working pattern for mixed Docker + Kubernetes environments.

Output Formats

alloygen host-01 --format alloy
alloygen host-01 --format configmap --namespace monitoring
alloygen --all --format both
alloygen --all --format argocd --argocd-repo-url [e-mail]:your/private-repo.git
alloygen --all --format all --argocd-repo-url [e-mail]:your/private-repo.git

Staging Hosts (Disable by Default)

Add enabled: false to a host to skip generation until it’s ready:

name: new-host
enabled: false

Generate disabled hosts explicitly:

alloygen --all --include-disabled

Cleaning Old Outputs

To remove generated configs for hosts that no longer exist:

alloygen --all --format both --clean

Deterministic Defaults

• No timestamp unless --include-timestamp
• ${VAR} placeholders preserved unless --resolve-env
• generated/manifest.json and generated/manifest.sha256 provide audit hashes

Private Values and Local Overrides

If you plan to open‑source this, keep real endpoints and generated outputs in a private repo. The public repo can contain only the generator, templates, and example definitions.

Deployment Notes

• Docker: mount the generated .alloy into the container.
• Kubernetes: --format configmap --namespace monitoring and apply the file.
• Argo CD: use --format argocd (or --format all) to generate per-cluster Application manifests.

Alerting (Starter Rules)

A minimal Prometheus ruleset lives at alerts/prometheus.rules.yaml. Import it into your Prometheus config to get basic “failure-first” alerts (host down, Alloy down, disk/memory pressure, kubelet scrape down).

Contributing

I’m happy to accept new scrape targets and improvements. To keep the project stable and safe for all users:

• Use example data only: add new scrapes/endpoints/hosts under definitions.example/ (no real domains, secrets, or internal IPs).
• Keep it deterministic: avoid timestamps, random values, or environment-only behavior in examples.
• Add tests: if you add a new scrape type or behavior, include/extend a fixture under tests/fixtures/basic/ and add a pytest to cover it.
• Prefer backward‑compatible changes: avoid breaking existing field names or defaults; if you must, document it clearly.
• Document the scrape: add a short note in README (or a comment in the example YAML) explaining what it does and when to use it.

Build/Test Prerequisites

Required for local build/test (make preflight):

• Python 3.9+ (3.11+ recommended)
• uv (package/env manager)
• make
• git
• alloy in PATH (used by Alloy config validation)

Additional required tools for full local CI parity (make verify):

• actionlint (workflow linting)
• act (run GitHub Actions locally)
• Docker running locally (required by act)

Run tests locally:

uv pip install -e ".[dev]"
uv run pytest -q

Pre-submit (one command):

make preflight

Comprehensive local gate (recommended before opening a PR):

make verify

Install local git hooks (recommended, runs version-bump guard on push):

make hooks-install

Optional local CI parity with act:

make preflight-act

Ways to contribute (beyond scrapes):

• Templates: alternate config templates or deployment styles.
• Output formats: Helm, Kustomize, Terraform, Secrets, etc.
• Validation rules: schema checks and better error messages.
• Example packs: curated definitions.example sets for common stacks.
• Docs: integration guides and usage patterns.
• Tests/fixtures: edge cases and new scrape types.
• Label conventions: standard label sets for common platforms/exporters.

Releases (PyPI)

1. Bump the version in pyproject.toml in your PR.
2. Open a PR — CI runs lint/tests/example checks and packaging smoke tests before merge.
3. The Release workflow promotes those artifacts, tags vX.Y.Z, publishes to PyPI, and creates a GitHub Release with auto‑generated notes.

Release notes live in GitHub Releases; PyPI will show the README and link back to GitHub for per‑version details.