esys-ci-host 0.3.8

Library and CLI to prepare Linux VPS hosts as GitLab CI runners

esys-ci-host

Library and CLI to prepare Linux VPS hosts as GitLab CI runners (Docker, privileged DinD, runner registration).

Extracted from the CI-host portions of esysdox-ops so LibESys superbuild and other tooling can depend on a focused package.

Install

pip install esys-ci-host

On a VPS, prefer the dedicated virtualenv used by remote mode (avoids apt/pip conflicts):

sudo apt-get update
sudo apt-get install -y python3-venv python3-pip python3.12-venv   # match your python3 -V
sudo mkdir -p /opt/esys-ci-host
sudo rm -rf /opt/esys-ci-host/venv
sudo python3 -m venv /opt/esys-ci-host/venv
sudo /opt/esys-ci-host/venv/bin/python -m ensurepip --upgrade
sudo /opt/esys-ci-host/venv/bin/python -m pip install esys-ci-host
sudo ln -sf /opt/esys-ci-host/venv/bin/esys-ci-host /usr/local/bin/esys-ci-host

Remote commands (esys-ci-host remote ...) create and use this venv automatically.

Updating

Upgrade esys-ci-host without touching pip paths manually. The command detects how the currently running CLI was installed and upgrades that environment.

Install type	Detected when	Upgrade method
Managed VPS	Package under /opt/esys-ci-host/venv	sudo + refresh /usr/local/bin/esys-ci-host
Virtualenv	CLI runs inside a Python venv	python -m pip install --upgrade in that venv
User install	Package under ~/.local	pip install --user --upgrade

On the VPS (as admin):

esys-ci-host update                  # latest from PyPI
esys-ci-host update --version 0.3.6  # specific version
esys-ci-host update --dry-run        # show detected install and planned pip command

Do not use sudo pip install on the VPS — that hits system Python and can conflict with apt.

From your laptop:

esys-ci-host remote --host 161.97.147.246 --user esysadmin --identity-file ~/.ssh/id_ed25519 update
esys-ci-host remote --host ... --user esysadmin --identity-file ... update --version 0.3.6

Option	Purpose
--version VERSION	Pin the PyPI version (default: latest)
--wheel PATH	Install from a local wheel instead of PyPI
--dry-run	Print detected install layout without upgrading

Remote update runs update on the VPS. On older installs without the update subcommand, it falls back to upgrading the managed virtualenv directly.

Quick start

On a fresh Ubuntu or Debian VPS (Docker executor for LibESys CI):

sudo esys-ci-host host bootstrap --yes --admin-user deploy --ssh-key-file ~/.ssh/id_ed25519.pub
# from your laptop: ssh deploy@<vps-ip>   (confirm login works)
sudo esys-ci-host host harden --yes
sudo esys-ci-host runner prepare --yes
esys-ci-host runner register --url https://gitlab.example.com --profile ci
esys-ci-host doctor

host bootstrap creates the admin user, installs OpenSSH, installs your SSH public key (or generates one under /etc/esys-ci-host/), verifies key setup, and enables UFW with SSH allowed. Root SSH stays enabled until you run host harden after confirming deploy login from your machine. Pass --ssh-key-file ~/.ssh/id_ed25519.pub or set ESYS_CI_HOST_ADMIN_SSH_PUBLIC_KEY / ESYS_CI_HOST_ADMIN_SSH_PUBLIC_KEY_FILE. Use --disable-root-ssh only if you accept lockout risk without external login verification.

The admin user has no login password (SSH key only). Bootstrap grants passwordless sudo via /etc/sudoers.d/99-esys-ci-host-<admin-user> so commands like sudo esys-ci-host host harden work after key login.

Set ESYS_CI_HOST_RUNNER_TOKEN instead of passing --token when registering.

Remote configuration (from your laptop)

Run the same host and runner steps over SSH without logging in manually first. Remote mode installs esys-ci-host into /opt/esys-ci-host/venv on the VPS (from PyPI or a local wheel), links /usr/local/bin/esys-ci-host, uploads your local SSH public key for the admin user, and runs the usual commands there.

Interactive (prompts for the root password when no key is given):

esys-ci-host remote --host 203.0.113.10 host bootstrap --yes --admin-user deploy
ssh -i ~/.ssh/id_ed25519 deploy@203.0.113.10   # verify login (admin has no password)
esys-ci-host remote --host 203.0.113.10 --user deploy --identity-file ~/.ssh/id_ed25519 host harden --yes
esys-ci-host remote --host 203.0.113.10 --user deploy --identity-file ~/.ssh/id_ed25519 runner prepare --yes

After a successful remote bootstrap, the tool can offer to write a managed ~/.ssh/config entry. It suggests a short name (for an IP like 161.97.147.246, default is {admin-user}-vps, e.g. esysadmin-vps; for runner01.example.com, default is runner01) and lets you change it before saving. Then ssh esysadmin-vps (or your chosen name) works. Use --setup-ssh-config to apply without prompts, --ssh-config-host NAME to set the alias directly, or --skip-ssh-config to disable the flow. esys-ci-host remote still needs --user and --identity-file today (Paramiko does not read OpenSSH config).

Non-interactive (CI or scripts — password via flag or env):

export ESYS_CI_HOST_SSH_PASSWORD='...'
esys-ci-host remote --host 203.0.113.10 --non-interactive host bootstrap --yes --admin-user deploy

Option	Purpose
--host HOST	Remote VPS hostname or IP (required)
--user USER	SSH login user (default: root)
--password PASS	SSH password (ESYS_CI_HOST_SSH_PASSWORD)
--identity-file PATH	SSH private key (ESYS_CI_HOST_SSH_IDENTITY_FILE)
--non-interactive	Fail instead of prompting for a password
--wheel PATH	Install from a local wheel instead of PyPI
--ssh-key-file PATH	Local admin public key (default: ~/.ssh/id_ed25519.pub)

Prefer --identity-file over --password after the first bootstrap. Avoid putting passwords on the command line in shared environments.

Commands still run on the VPS when you SSH in directly (host bootstrap, runner prepare, etc.) — both models are supported.

Host bootstrap

Option	Purpose
--admin-user NAME	Non-root login account with sudo (default: esysadmin)
--ssh-key KEY	OpenSSH public key line for admin login
--ssh-key-file PATH	File containing the public key
--ssh-port PORT	SSH port used for localhost login verification (default: 22)
--disable-root-ssh	Disable root SSH during bootstrap (default: keep root until host harden)
--yes	Required to apply host changes
--dry-run	Print steps without mutating the host

Admin login is SSH key only (no account password). Bootstrap writes a sudoers drop-in so the admin user can run sudo without a password.

If no key is supplied, an ed25519 key pair is generated at /etc/esys-ci-host/<admin-user>-id_ed25519 — copy the private key securely before closing your root session.

After bootstrap: SSH in as the admin user from your machine, then run sudo esys-ci-host host harden --yes to disable root login.

Runner registration and executors

The GitLab executor is chosen at registration time (runner register), not during runner install or runner prepare.

Option	Purpose
--executor shell	Run jobs directly on the host (default)
--executor docker	Run jobs in containers (requires runner prepare)
--profile ci	LibESys CI preset: docker executor, privileged DinD, tags docker-privileged, and a sensible default image
--docker-image IMAGE	Override the runner’s default image (fallback only; see note below)
--docker-privileged	Privileged docker executor (enabled automatically by --profile ci)
--tags TAGS	Comma-separated runner tags (e.g. linux,docker)

Default image vs job image: GitLab Runner’s --docker-image (at register time) is only the fallback written into config.toml. Each CI job uses its own image: from .gitlab-ci.yml when present — you do not need to pass --docker-image for every image your pipeline uses. With --profile ci, a default (python:3.11-bookworm) is applied automatically; override it only if you want a different fallback for jobs that omit image:.

Shell executor (simple jobs on the host):

sudo esys-ci-host runner install --yes
esys-ci-host runner register --url https://gitlab.example.com --executor shell --tags shell

Docker executor (explicit; run prepare first). GitLab Runner requires a default image for non-interactive docker registration, so pass --docker-image here unless you use --profile ci:

sudo esys-ci-host runner prepare --yes
esys-ci-host runner register \
  --url https://gitlab.example.com \
  --executor docker \
  --docker-image python:3.11-bookworm \
  --docker-privileged \
  --tags docker-privileged

LibESys CI profile (recommended; includes docker executor defaults — no need to pass --docker-image unless overriding the fallback):

runner prepare creates the gitlab-admin group, adds the bootstrap admin (and gitlab-runner for service access), and makes /etc/gitlab-runner group-writable so runner register can patch config.toml without sudo for every read/write. Register and systemctl still use sudo when needed.

sudo esys-ci-host runner prepare --yes
esys-ci-host runner register --url https://gitlab.example.com --profile ci

Use --dry-run on runner register to print the gitlab-runner register command without executing it.

Development

Platform-specific host operations (packages, users, firewall, SSH) live under src/esys_ci_host/platform/. DebianAptBackend is implemented today; add new HostPlatformBackend subclasses (for example RhelDnfBackend) and register them in platform/registry.py to support additional distributions later. Runner install/prepare still use apt directly and will migrate to the platform layer in a follow-up.

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pre-commit install
pre-commit install --hook-type commit-msg
pytest

Release tags vX.Y.Z must match src/esys_ci_host/__version__.py. Never tag until master has a green pipeline on the commit you are releasing.

1. On develop: make bump-release (or python scripts/bump_release.py --yes --version X.Y.Z), then push and wait for green CI.
2. Fast-forward master from develop, push, and wait for green CI on master.
3. On master: make tag-release (or python scripts/tag_release.py --yes), then git push origin vX.Y.Z.

CI publishes tagged releases to GitLab PyPI and TestPyPI automatically; public PyPI is manual (publish:pypi).

Integration tests (Docker + systemd, Ubuntu 24.04 by default):

# Full runner workflow
bash tests/integration/run_docker.sh

# Host bootstrap + harden workflow
INTEGRATION_SCRIPT=/usr/local/bin/run-integration-host-bootstrap bash tests/integration/run_docker.sh

# Windows
.\test.ps1 -HostBootstrap

License

Apache-2.0 — see LICENSE.