compose-farm 0.4.0

Compose Farm - run docker compose commands across multiple hosts

• Compose Farm
  • Why Compose Farm?
  • Key Assumption: Shared Storage
  • Installation
  • Configuration
  • Usage
  • Traefik Multihost Ingress (File Provider)
  • Requirements
  • How It Works
  • License

Compose Farm

A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.

Why Compose Farm?

I run 100+ Docker Compose stacks on an LXC container that frequently runs out of memory. I needed a way to distribute services across multiple machines without the complexity of:

• Kubernetes: Overkill for my use case. I don't need pods, services, ingress controllers, or YAML manifests 10x the size of my compose files.
• Docker Swarm: Effectively in maintenance mode—no longer being invested in by Docker.

Compose Farm is intentionally simple: one YAML config mapping services to hosts, and a CLI that runs docker compose commands over SSH. That's it.

Key Assumption: Shared Storage

Compose Farm assumes all your compose files are accessible at the same path on all hosts. This is typically achieved via:

• NFS mount (e.g., /opt/compose mounted from a NAS)
• Synced folders (e.g., Syncthing, rsync)
• Shared filesystem (e.g., GlusterFS, Ceph)

# Example: NFS mount on all hosts
nas:/volume1/compose  →  /opt/compose (on nas01)
nas:/volume1/compose  →  /opt/compose (on nas02)
nas:/volume1/compose  →  /opt/compose (on nas03)

Compose Farm simply runs docker compose -f /opt/compose/{service}/docker-compose.yml on the appropriate host—it doesn't copy or sync files.

Installation

pip install compose-farm
# or
uv pip install compose-farm

Configuration

Create ~/.config/compose-farm/compose-farm.yaml (or ./compose-farm.yaml in your working directory):

compose_dir: /opt/compose  # Must be the same path on all hosts

hosts:
  nas01:
    address: 192.168.1.10
    user: docker
  nas02:
    address: 192.168.1.11
    # user defaults to current user
  local: localhost  # Run locally without SSH

services:
  plex: nas01
  jellyfin: nas02
  sonarr: nas01
  radarr: local  # Runs on the machine where you invoke compose-farm

Compose files are expected at {compose_dir}/{service}/docker-compose.yml.

Usage

# Start services
compose-farm up plex jellyfin
compose-farm up --all

# Stop services
compose-farm down plex

# Pull latest images
compose-farm pull --all

# Restart (down + up)
compose-farm restart plex

# Update (pull + down + up) - the end-to-end update command
compose-farm update --all

# Capture image digests to a TOML log (per service or all)
compose-farm snapshot plex
compose-farm snapshot --all  # writes ~/.config/compose-farm/dockerfarm-log.toml

# View logs
compose-farm logs plex
compose-farm logs -f plex  # follow

# Show status
compose-farm ps

Traefik Multihost Ingress (File Provider)

If you run a single Traefik instance on one “front‑door” host and want it to route to Compose Farm services on other hosts, Compose Farm can generate a Traefik file‑provider fragment from your existing compose labels.

How it works

• Your docker-compose.yml remains the source of truth. Put normal traefik.* labels on the container you want exposed.
• Labels and port specs may use ${VAR} / ${VAR:-default}; Compose Farm resolves these using the stack’s .env file and your current environment, just like Docker Compose.
• Publish a host port for that container (via ports:). The generator prefers host‑published ports so Traefik can reach the service across hosts; if none are found, it warns and you’d need L3 reachability to container IPs.
• If a router label doesn’t specify traefik.http.routers.<name>.service and there’s only one Traefik service defined on that container, Compose Farm wires the router to it.
• compose-farm.yaml stays unchanged: just hosts and services: service → host.

Example docker-compose.yml pattern:

services:
  plex:
    ports: ["32400:32400"]
    labels:
      - traefik.enable=true
      - traefik.http.routers.plex.rule=Host(`plex.lab.mydomain.org`)
      - traefik.http.routers.plex.entrypoints=websecure
      - traefik.http.routers.plex.tls.certresolver=letsencrypt
      - traefik.http.services.plex.loadbalancer.server.port=32400

One‑time Traefik setup

Enable a file provider watching a directory (any path is fine; a common choice is on your shared/NFS mount):

providers:
  file:
    directory: /mnt/data/traefik/dynamic.d
    watch: true

Generate the fragment

compose-farm traefik-file --output /mnt/data/traefik/dynamic.d/compose-farm.generated.yml

Re‑run this after changing Traefik labels, moving a service to another host, or changing published ports.

Requirements

• Python 3.11+
• SSH key-based authentication to your hosts (uses ssh-agent)
• Docker and Docker Compose installed on all target hosts
• Shared storage: All compose files at the same path on all hosts (NFS, Syncthing, etc.)

How It Works

1. You run compose-farm up plex
2. Compose Farm looks up which host runs plex (e.g., nas01)
3. It SSHs to nas01 (or runs locally if localhost)
4. It executes docker compose -f /opt/compose/plex/docker-compose.yml up -d
5. Output is streamed back with [plex] prefix

That's it. No orchestration, no service discovery, no magic.

License

MIT