Multi-Registry Pull Through Cache 🚀

Generate a complete Docker Compose stack with pull-through caches for multiple container registries, fronted by Traefik and accelerated by Redis.

⚠️ Migrating from the script-based version? See the upgrade guide. TL;DR: python setup.py → multi-registry-cache setup, python generate.py → multi-registry-cache generate. Your existing config.yaml and Docker usage are fully compatible.

---

flowchart TB
    subgraph Clients
        D[Docker / containerd]
        K[Kubernetes nodes]
    end

    subgraph Cache Stack
        T[Traefik Proxy\nTLS + routing]
        R1[Registry: docker.io]
        R2[Registry: ghcr.io]
        R3[Registry: quay.io]
        RN[Registry: ...]
        RE[Redis\nblob cache]
    end

    subgraph Upstream
        U1[(Docker Hub)]
        U2[(GHCR)]
        U3[(Quay)]
        UN[(...)]
    end

    D & K -->|pull| T
    T --> R1 & R2 & R3 & RN
    R1 & R2 & R3 & RN --> RE
    R1 -.->|cache miss| U1
    R2 -.->|cache miss| U2
    R3 -.->|cache miss| U3
    RN -.->|cache miss| UN

---

🚀 Features

Feature	Description
🏗️ Multi-Registry	Mirror Docker Hub, GHCR, Quay, NVCR, or any OCI registry
🔒 Private Registry	Add a private registry alongside your caches
🌐 Traefik Proxy	Automatic TLS termination and subdomain-based routing
⚡ Redis Acceleration	Blob descriptor caching for faster repeated pulls
🧙 Interactive Wizard	Guided setup command to generate your config.yaml
📦 Installable CLI	uvx, pipx, pip install, or Docker — your choice
☸️ K8s Ready	Works with k3s, RKE2, containerd, dockerd
🗄️ Flexible Storage	Filesystem, S3, GCS, or in-memory backends
🔧 Customizable	Full control over Traefik, registry, and Compose config via {name} interpolation
🌱 Eco-Friendly	Less external bandwidth = smaller carbon footprint

---

📦 Installation

With uvx (recommended, zero install)

uvx multi-registry-cache setup
uvx multi-registry-cache generate

With uv tool / pipx

uv tool install multi-registry-cache   # or: pipx install multi-registry-cache
multi-registry-cache setup
multi-registry-cache generate

With pip

pip install multi-registry-cache

With Docker

touch config.yaml
docker run --rm -ti -v "./config.yaml:/app/config.yaml" obeoneorg/multi-registry-cache setup
docker run --rm -ti -v "./config.yaml:/app/config.yaml" -v "./compose:/app/compose" obeoneorg/multi-registry-cache generate

---

🛠️ Quick Start

flowchart LR
    A["1️⃣ setup"] --> B["2️⃣ edit config"] --> C["3️⃣ generate"] --> D["4️⃣ docker compose up"]

1. Run the interactive wizard

multi-registry-cache setup

2. Review & fine-tune

Edit config.yaml for advanced settings (TLS, Let's Encrypt, storage backends, etc.). See CONFIG.md for details.

3. Generate the stack

multi-registry-cache generate

4. Launch

cd compose && docker compose up -d

---

⚙️ CLI Reference

Command	Description
multi-registry-cache setup	Interactive wizard → config.yaml
multi-registry-cache generate	Read config.yaml → generate compose/
multi-registry-cache --help	Show all commands

Option	Applies to	Default	Description
--config, -c	setup, generate	config.yaml	Config file path
--output-dir, -o	generate	compose	Output directory

---

🔄 Configuring Container Runtimes

containerd

[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
    endpoint = ["https://dockerhub.registry-cache.example.net"]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."ghcr.io"]
    endpoint = ["https://ghcr.registry-cache.example.net"]

sudo systemctl restart containerd

nerdctl

mkdir -p /etc/containerd/certs.d/docker.io/

/etc/containerd/certs.d/docker.io/hosts.toml:

server = "https://docker.io"

[host."https://dockerhub.registry-cache.example.net"]
  capabilities = ["pull", "resolve"]

dockerd

{
  "registry-mirrors": ["https://dockerhub.registry-cache.example.net"]
}

sudo systemctl daemon-reload && sudo systemctl restart docker

k3s / RKE2

/etc/rancher/(k3s|rke2)/registries.yaml:

mirrors:
  docker.io:
    endpoint:
      - https://dockerhub.registry-cache.example.net
  ghcr.io:
    endpoint:
      - https://ghcr.registry-cache.example.net

BuildKit

BuildKit resolves images independently from dockerd. Without explicit configuration, docker build will not use your registry mirrors.

Create a buildkitd.toml with your mirrors:

[registry."docker.io"]
  mirrors = ["dockerhub.registry-cache.example.net"]

[registry."ghcr.io"]
  mirrors = ["ghcr.registry-cache.example.net"]

[registry."dockerhub.registry-cache.example.net"]
  http = false
  # ca = ["/etc/certs/my-ca.pem"]  # if using a private CA

[registry."ghcr.registry-cache.example.net"]
  http = false

Where to place this file depends on your setup:

Context	Config path
Docker Engine (rootful)	/etc/buildkit/buildkitd.toml
Docker Engine (rootless)	~/.config/buildkit/buildkitd.toml
buildx auto-detection	~/.docker/buildx/buildkitd.default.toml
Docker Desktop default builder	Not supported (see below)

Docker Desktop: the default builder uses the docker driver, which does not support buildkitd.toml. You must create a separate builder with the docker-container driver:

docker buildx create --use --bootstrap \
  --name cached-builder \
  --driver docker-container \
  --buildkitd-config /path/to/buildkitd.toml

Other Kubernetes distributions

Configure each node's container runtime to use the registry mirror. Refer to your distribution's documentation.

---

📄 License

MIT — Grégoire Compagnon

---

Contributions welcome! Open an issue or submit a PR. 🐋