mirror.py

A Linux daemon for maintaining local mirrors of remote package repositories. Schedule synchronization, track each repository, and inspect logs through one master-worker service.

Getting started · Configuration · CLI reference · Documentation

Features

• Scheduled and push-triggered syncs. Configure intervals per repository, retry failed jobs, or trigger a sync from the command line.
• Separate scheduling and execution. The master manages schedules and state; the worker runs subprocesses with configured UID/GID and keeps active jobs running when the master restarts.
• Status and logs. Inspect packages with the terminal UI, keep per-run logs with gzip compression, and publish status as JSON.
• Configuration reloads. Apply supported configuration changes to the running daemon without restarting it.
• Standalone execution. Run a single sync in the foreground without starting the daemon and worker services.
• Extensible plugins. Add sync methods, event handlers, and status fields through Python entry points.

Supported sync methods

See each method's documentation for prerequisites, configuration examples, selection rules, and signature verification requirements.

Installation

Requires Linux and Python 3.10 or later.

Install from PyPI

Install mirror.py globally so the mirror command is available to root and systemd provisioning:

sudo python3 -m pip install mirror.py
sudo mirror --version

For apt-mirror2 support, install the optional extra globally as well:

sudo python3 -m pip install 'mirror.py[apt-mirror2]'

Some distributions mark their system Python as externally managed and reject global pip installs. In that case, use the virtual-environment alternative in the installation guide instead of overriding that protection.

Install from source

For development or features not yet released on PyPI, use uv:

git clone https://github.com/sparcs-kaist/mirror.py.git
cd mirror.py
uv sync
source .venv/bin/activate
mirror --version

To include apt-mirror2 in a source installation, run uv sync --extra apt-mirror2. This README describes the source tree; an installed release may differ. See the installation guide for more details.

System dependencies

Install the system tools required by your chosen sync methods. mirror setup checks for all three of rsync, lftp, and bandersnatch. Bandersnatch is installed with the Python package. For example, on Debian or Ubuntu, install the other two with:

sudo apt install rsync lftp

Signed APT repositories require the relevant verification tools and trusted keyrings described in the debmirror and apt-mirror2 guides.

Quickstart

1. Provision the host

The commands below assume the recommended global installation. For a source or virtual-environment installation, activate the environment and run every root CLI command through it, for example sudo env "PATH=$PATH" mirror setup. Use the same prefix for manual worker, daemon, tui, and config reload calls.

sudo mirror setup

Setup creates the configuration, state, socket, log, and web directories, and installs mirror.service and mirror-worker.service. It creates /etc/mirror/config.json only when that file does not already exist. It also installs Bash completion at /usr/local/share/bash-completion/completions/mirror. Completion requires Bash 4.4 or later and the distribution's bash-completion package to be installed and enabled; setup does not install that package or edit per-user shell files. Open a new shell, then type mirror t and press Tab to complete mirror tui.

2. Configure a repository

Edit /etc/mirror/config.json. Set the mirror identity, maintainer details, local timezone, and the non-root settings.uid / settings.gid used by sync subprocesses. Replace the empty packages object with entries for your mirrors. For example, this is a packages value for an rsync mirror:

{
  "repository": {
    "id": "repository",
    "name": "Example repository",
    "href": "/repository",
    "synctype": "rsync",
    "syncrate": "PT6H",
    "link": [],
    "settings": {
      "hidden": false,
      "src": "rsync://upstream.example.org/repository/",
      "dst": "/srv/mirror/repository",
      "options": {}
    }
  }
}

Replace the placeholder upstream with your actual source. PT6H means every six hours. Create the destination directory and grant the configured UID/GID write access before starting the services. Sync methods can delete obsolete files, so use a directory dedicated to the mirror.

The quickstart guide provides a full configuration. See the configuration reference and example configuration for more options. The browser configuration editor can also generate configuration files; it does not connect to or configure the daemon directly.

3. Start the worker and master

Use the systemd units installed by mirror setup. Enable both services at boot, start them, and check their status:

sudo systemctl enable --now mirror-worker.service mirror.service
sudo systemctl status mirror-worker.service mirror.service

Both services read /etc/mirror/config.json. The master schedules syncs and communicates with the worker over Unix sockets. See installation for deployment details.

4. Inspect and control syncs

sudo mirror tui
sudo mirror config reload

The TUI shows package status and logs. push requests an immediate sync; config reload applies supported changes after you edit the configuration. Settings that require a restart are reported as warnings.

For one-off jobs, use mirror standalone SYNCTYPE. See the CLI reference for arguments, options, and TUI key bindings.

Configuration, state, and serving files

Repository files are stored in each package's settings.dst. Configure your own HTTP, FTP, or rsync server to publish those files and, if needed, the status JSON. mirror.py handles synchronization and status generation; serving the mirror is a separate deployment step.

See state files, architecture, and troubleshooting for operational details.

Development

uv sync
uv run pytest
uv run pytest -m integration -v

The default pytest run excludes integration tests. The integration suite needs Docker with Compose and builds containers using the current source tree. Run the suites sequentially; see the integration guide for host requirements and fixture behavior.

For documentation and configuration editor builds, see Contributing. For extensions, see the plugin author guide and the example plugin.

Report bugs and feature requests through GitHub Issues.

License

Apache License 2.0. Maintained by SPARCS at KAIST.

Contact: ftp@ftp.kaist.ac.kr.