duplicacy-exporter 0.4.0

Prometheus exporter for Duplicacy backup metrics -- real-time progress, speed, and post-run summaries

Prometheus exporter for Duplicacy backup metrics -- real-time progress, speed, and post-run summaries for your Grafana dashboards.

---

Overview

duplicacy-exporter bridges Duplicacy backups and Prometheus, giving you full observability over backup operations. It works with both Duplicacy CLI (via log tailing) and Duplicacy Web UI (via webhook), exposing metrics that Prometheus scrapes and Grafana visualizes.

Key capabilities

• Real-time metrics -- backup speed (bytes/sec), progress (0-100%), chunks uploaded/skipped, updated per chunk
• Post-run summaries -- duration, file counts, bytes uploaded, exit codes, revision numbers
• Prune tracking -- monitors prune operations with completion timestamps
• Two collection modes -- log_tail for CLI users, webhook for Web UI users
• Smart label resolution -- automatic snapshot ID, storage target, and machine name detection from logs
• Storage host mapping -- translates IPs and Tailscale FQDNs into human-readable names
• Lightweight -- single Python file, one dependency (prometheus_client), Alpine image (~30 MB)

Architecture

+---------------------+         +----------------------+         +------------+
|                     |  logs   |                      | scrape  |            |
|  Duplicacy CLI      +-------->+  duplicacy-exporter  +<--------+ Prometheus |
|  (Docker / file)    |  tail   |                      |  :9750  |            |
+---------------------+         +----------+-----------+         +------+-----+
                                           |                            |
+---------------------+  POST   |          |                            |
|  Duplicacy Web UI   +-------->+  /webhook endpoint   |         +------v-----+
|  (report_url)       |         |                      |         |  Grafana   |
+---------------------+         +----------------------+         +------------+

Log tail mode connects to the Docker Engine API over a Unix socket (or tails a log file) and parses Duplicacy output line-by-line. It extracts chunk-level progress in real time and summary statistics at completion.

Webhook mode receives JSON payloads from Duplicacy Web UI's report_url setting, extracting the same summary metrics without needing Docker socket access.

Quick Start

Docker Compose -- Log Tail Mode (recommended for CLI)

Deploy alongside your Duplicacy CLI container using a shared log volume:

services:
  duplicacy-exporter:
    image: drumsergio/duplicacy-exporter:0.4.0
    container_name: duplicacy-exporter
    restart: unless-stopped
    environment:
      - MODE=log_tail
      - LOG_FILE=/logs/duplicacy.log
      - LISTEN_PORT=9750
    volumes:
      - duplicacy-logs:/logs:ro
    ports:
      - "9750:9750"

volumes:
  duplicacy-logs:

Note: Mount the same duplicacy-logs volume in your Duplicacy container, writing output to /logs/duplicacy.log. This avoids exposing the Docker socket.

Docker Compose -- Webhook Mode (for Web UI)

services:
  duplicacy-exporter:
    image: drumsergio/duplicacy-exporter:0.4.0
    container_name: duplicacy-exporter
    restart: unless-stopped
    environment:
      - MODE=webhook
      - LISTEN_PORT=9750
    ports:
      - "9750:9750"

Then set report_url in Duplicacy Web UI to: http://duplicacy-exporter:9750/webhook

Docker Compose -- Log File Mode

If you write Duplicacy logs to a file instead of using Docker:

services:
  duplicacy-exporter:
    image: drumsergio/duplicacy-exporter:0.4.0
    container_name: duplicacy-exporter
    restart: unless-stopped
    environment:
      - MODE=log_tail
      - LOG_FILE=/logs/duplicacy.log
      - LISTEN_PORT=9750
    volumes:
      - /path/to/duplicacy/logs:/logs:ro
    ports:
      - "9750:9750"

Configuration

All configuration is done through environment variables:

Variable	Default	Description
MODE	log_tail	Collection mode: log_tail or webhook
DOCKER_CONTAINER_NAME	duplicacy-cli-cron	Container name to tail logs from (log_tail mode)
LOG_FILE	(empty)	Path to log file; alternative to Docker socket (log_tail mode)
LISTEN_PORT	9750	Port for the metrics and webhook HTTP server
WEBHOOK_PATH	/webhook	Path for the webhook POST endpoint (webhook mode)
MACHINE_NAME	(empty)	Machine name label. In log_tail mode it must be set (or learned from a DUPLICACY_META / notification line) before backup metrics are emitted.
SNAPSHOT_ID	(empty)	Snapshot id for log_tail users whose logs have no section headers / DUPLICACY_META (e.g. stock duplicacy backup). Lets post-run summary metrics resolve.
TAILSCALE_DOMAIN	mango-alpha.ts.net	Tailscale domain suffix to strip from storage URLs
STORAGE_HOST_MAP	(empty)	JSON object mapping hostname/IP to display name
REPLAY_HOURS	25	Hours of Docker log history to replay on startup
TIMESTAMP_FILE	/tmp/duplicacy_exporter_last_ts	File to persist last-seen log timestamp (avoids counter double-count on restart)
MAX_LOG_BUFFER	1048576	Maximum Docker log buffer size in bytes before discarding partial data (1 MB)
LOG_LEVEL	INFO	Logging verbosity: DEBUG, INFO, WARNING, ERROR

Storage host mapping example

Map raw IPs or hostnames to friendly names:

STORAGE_HOST_MAP='{"192.168.10.100":"watchtower","192.168.20.5":"geiserct"}'

Metrics

All backup metrics carry labels: snapshot_id, storage_target, machine. All prune metrics carry labels: storage_target, machine.

Each distinct (snapshot_id, storage_target) is its own series (and its own device in duplicacy-ha), so multiple backups are tracked independently. In webhook mode snapshot_id comes from the Web UI report's backup id, falling back to the source directory name — so two backups on one machine never collapse into one. In log_tail mode it comes from a DUPLICACY_META snapshot_id=… line, a --- Backup -> … (id) --- section header, or the SNAPSHOT_ID env var.

Real-time (updated per chunk during backup)

Metric	Type	Description
duplicacy_backup_running	Gauge	1 if backup is in progress, 0 otherwise
duplicacy_backup_speed_bytes_per_second	Gauge	Current backup speed
duplicacy_backup_progress_ratio	Gauge	Progress from 0.0 to 1.0
duplicacy_backup_chunks_uploaded	Gauge	Chunks uploaded in current run
duplicacy_backup_chunks_skipped	Gauge	Chunks skipped in current run

Post-run summary

Metric	Type	Description
duplicacy_backup_last_success_timestamp_seconds	Gauge	Unix timestamp of last successful backup
duplicacy_backup_last_duration_seconds	Gauge	Duration of last backup in seconds
duplicacy_backup_last_files_total	Gauge	Total files in last backup
duplicacy_backup_last_files_new	Gauge	New files in last backup
duplicacy_backup_last_bytes_uploaded	Gauge	Bytes uploaded in last backup
duplicacy_backup_last_bytes_new	Gauge	New bytes in last backup
duplicacy_backup_last_chunks_new	Gauge	New chunks in last backup
duplicacy_backup_last_exit_code	Gauge	Exit code: 0 = success, 1 = failure
duplicacy_backup_last_revision	Gauge	Revision number of last backup
duplicacy_backup_bytes_uploaded_total	Counter	Cumulative bytes uploaded across all runs

Prune

Metric	Type	Description
duplicacy_prune_running	Gauge	1 if prune is in progress
duplicacy_prune_last_success_timestamp_seconds	Gauge	Unix timestamp of last successful prune

Diagnostics

Metric	Type	Description
duplicacy_exporter_last_activity_timestamp_seconds	Gauge	Unix timestamp of the last log line parsed (alert if it goes stale)
duplicacy_exporter_backups_seen_total	Counter	Completed backups detected, including any dropped for a missing snapshot_id/storage_target/machine. If this climbs while labelled series stay empty, the exporter is seeing backups it can't label — set SNAPSHOT_ID/MACHINE_NAME.

Endpoints

Path	Method	Description
/metrics	GET	Prometheus metrics endpoint
/webhook	POST	Duplicacy Web UI report endpoint
/health	GET	Health check (returns 200 OK)

Prometheus Configuration

Add the exporter as a scrape target:

scrape_configs:
  - job_name: "duplicacy"
    static_configs:
      - targets: ["duplicacy-exporter:9750"]
        labels:
          instance: "my-server"

Example alerting rule

groups:
  - name: duplicacy
    rules:
      - alert: DuplicacyBackupFailed
        expr: duplicacy_backup_last_exit_code != 0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Duplicacy backup failed for {{ $labels.snapshot_id }}"

      - alert: DuplicacyBackupStale
        expr: time() - duplicacy_backup_last_success_timestamp_seconds > 86400
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: "No successful Duplicacy backup in 24h for {{ $labels.snapshot_id }}"

Grafana Dashboard

A ready-to-import dashboard is included in dashboard.json and published on Grafana.com (#25089).

Import it in Grafana via Dashboards → Import → Upload JSON file or use the dashboard ID 25089.

Troubleshooting

Exporter starts but no metrics appear

• Log tail mode: Verify the Docker socket is mounted (/var/run/docker.sock:/var/run/docker.sock:ro) and the DOCKER_CONTAINER_NAME matches your Duplicacy container exactly.
• Log file mode: Confirm the log file path is correct and the volume mount provides read access.
• Webhook mode: Ensure report_url in Duplicacy Web UI points to http://<exporter-host>:9750/webhook. The exporter must be reachable from the Web UI container.

Metrics show but labels are wrong or missing

• Set LOG_LEVEL=DEBUG to see how each log line is parsed and which labels are resolved.
• If storage targets show as raw IPs, use STORAGE_HOST_MAP to map them to friendly names.
• If machine name is missing, set MACHINE_NAME explicitly.

Docker socket permission denied

The exporter process runs as root inside the container by default. If you run it as a non-root user, ensure the user has access to the Docker socket (typically group docker, GID 999 or similar).

Webhook returns 404

Verify the WEBHOOK_PATH environment variable matches the path you configured in Duplicacy Web UI. The default is /webhook.

Related Projects

• duplicacy-container -- Runtime image and Helm chart for the Kubernetes Duplicacy stack
• duplicacy-cli-cron -- Scripts, wrappers, and backup recipes for Duplicacy CLI
• duplicacy-ha -- Home Assistant integration for Duplicacy
• duplicacy-mcp -- MCP server for Duplicacy
• Duplicacy -- Lock-free deduplication cloud backup tool
• Prometheus -- Monitoring and alerting toolkit
• Grafana -- Observability and visualization platform
• awesome-prometheus -- Curated list of Prometheus resources
• prometheus_client (Python) -- Official Prometheus client library for Python

License

GPL-3.0