gitlab-proxmox-runner 0.0.1

GitLab Runner Custom Executor that provisions a Proxmox VM and runs the job script inside it

gitlab-proxmox-runner

A GitLab Runner Custom Executor driver that provisions a fresh Proxmox VM for each CI job and runs the job's script inside that VM. CI variables (CI_JOB_ID, CI_COMMIT_SHA, …) are visible to user scripts the same way the docker executor exposes them.

The package ships four console scripts: vm_config_exec, vm_prepare_exec, vm_run_exec, vm_cleanup_exec. They map 1:1 to the four hooks of the gitlab-runner custom executor protocol.

How it works

Per CI job, gitlab-runner invokes the four entry points this package installs:

Entry point	Stage	Responsibility
vm_config_exec	once, first	Emit driver/build dirs/job_env JSON; resolves the requested template ID.
vm_prepare_exec	once	Clone the Proxmox template, boot the VM, wait for the QEMU guest agent, save {vmid, ip} state.
vm_run_exec	per stage	scp the runner-generated stage script to the VM, then ssh <user>@vm sudo /bin/bash …. Runs prepare_script, get_sources, build_script, upload_artifacts_*, etc., all in the VM.
vm_cleanup_exec	once, last	Best-effort stop + destroy of the VM, remove state file.

Per-job state is persisted to /tmp/gitlab-proxmox-runner-${CI_JOB_ID}.json so each stage can find the VM that was provisioned in vm_prepare_exec.

Templates

Pick one via PROXMOX_TEMPLATE (job-level variable):

Template name	Proxmox VMID
ubuntu-focal	60900
ubuntu-bionic	60910
debian-11	60920
debian-12	60930

Prerequisite: these template VMs must already exist on the Proxmox server at the matching VMIDs before any CI job runs. The driver only clones an existing template — it does not create one. The VMID column above is the ID the driver looks up in pve.<node>.qemu/<vmid>. If you use different VMIDs in your environment, fork this repo and edit VM_Templates in gitlab_proxmox_runner/__init__.py.

Each template VM must:

• have the QEMU guest agent installed and enabled,
• contain the SSH user named in VM_SSH_USER (default ci) with passwordless sudo and the runner's SSH public key in ~/.ssh/authorized_keys,
• be marked as a template in Proxmox (qm template <vmid>).

Required environment on the runner host

Set in [[runners]].environment (config.toml). gitlab-runner automatically prefixes build variables with CUSTOM_ENV_ when handing them to custom executor scripts, so write the names below without the prefix in config.toml — the executor reads them as CUSTOM_ENV_<NAME>.

Variable in config.toml	Read by executor as	Purpose
PROXMOX_HOST	CUSTOM_ENV_PROXMOX_HOST	Proxmox API host
PROXMOX_USER	CUSTOM_ENV_PROXMOX_USER	Proxmox user (e.g. root@pam)
PROXMOX_TOKEN_NAME	CUSTOM_ENV_PROXMOX_TOKEN_NAME	Proxmox API token name
PROXMOX_TOKEN_VALUE	CUSTOM_ENV_PROXMOX_TOKEN_VALUE	Proxmox API token value
PROXMOX_NODE	CUSTOM_ENV_PROXMOX_NODE	Proxmox node name to clone on
VM_SSH_USER	CUSTOM_ENV_VM_SSH_USER	SSH user inside the VM (default: ci)

VM_SSH_PRIVATE_KEY carries the private key for ssh/scp into the VM. It is typically a GitLab CI variable (instance- or project-level, File or Variable type). gitlab-runner exposes it to the executor as CUSTOM_ENV_VM_SSH_PRIVATE_KEY. The driver also accepts VM_SSH_PRIVATE_KEY (no prefix) for setups where the value is exported via the runner's systemd unit. The variable may contain either:

• the actual private-key text (the driver materialises it to a 0600 temp file on the runner host), or
• a filesystem path to an existing key file.

Installation on the runner host

sudo /usr/bin/pip install -U --break-system-packages gitlab-proxmox-runner

--break-system-packages is required on Debian 12 / Python 3.12+ where the system pip enforces PEP 668.

The four entry points install to /usr/local/bin/{vm_config_exec,vm_prepare_exec,vm_run_exec,vm_cleanup_exec}.

Runner config

A worked config.toml example lives at examples/config.toml. Copy the relevant [[runners]] block onto your runner host and systemctl restart gitlab-runner.

Using it from a job

The job script is plain shell — no manual SSH/SCP plumbing:

my-job:
  tags: [proxmox]
  variables:
    PROXMOX_TEMPLATE: ubuntu-focal
  script:
    - apt-get update && apt-get install -y build-essential
    - make
  artifacts:
    paths: [out/]

get_sources, the user script:, and upload_artifacts_* all execute inside the VM. CI variables are exported automatically. Artifacts are uploaded by gitlab-runner's normal flow.

Development

python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
ruff check .
pytest -q --cov=gitlab_proxmox_runner --cov-report=term-missing
python -m build      # produces sdist + wheel under dist/

The unit-test suite does not require a live Proxmox instance — Proxmox API calls and SSH/SCP subprocess calls are mocked out. End-to-end testing must be done on a host with network access to a Proxmox cluster.

Releasing to PyPI

CI publishes on tag pushes matching vX.Y.Z via PyPI Trusted Publishing (OIDC). One-time PyPI setup: at https://pypi.org/manage/account/publishing/ add a publisher with:

• Project name: gitlab-proxmox-runner
• Owner: your gitlab.com namespace
• Repository name: this repo's slug
• Workflow filename: .gitlab-ci.yml

Then git tag v0.1.0 && git push --tags triggers the publish job.

License

MIT — see LICENSE.