fherma-runner

Takes measurement jobs from a FHERMA platform and reports what happened.

uv tool install fherma-runner        # or: pipx install fherma-runner
fherma-runner --api https://fherma.io --name bench-01 --token fhr_...

A command line tool belongs in an environment of its own, and on macOS it has no choice: the system Python has no pip on the path at all, and Homebrew's is marked externally managed so installing into it is refused. uv tool and pipx both make that environment and leave fherma-runner on the path, which is the part pip install --user does not do.

Neither installed? One line, and nothing else on the machine changes:

curl -LsSf https://astral.sh/uv/install.sh | sh

Check what you got:

$ fherma-runner --version
fherma-runner 0.3.0 · python 3.13.4

The token comes from the platform, is good for an hour and is spent once: the first start exchanges it for this runner's own key and keeps that in ~/.fherma/runners/<name>/key. Every start after the first needs no token.

--api is the API, not the website. They are usually different hosts, and a website answers an unknown path with a page and a 200 rather than an error — so the wrong one here fails as "answered with something that is not JSON".

Every start says what the machine is now, so a server that was re-provisioned is matched on what it is rather than on what it was.

Several on one machine

A machine can hold more than one: a card that is free while the CPU is busy, or two cards, or one runner kept for reference work and one for everything else. Each is its own runner on the platform, with its own invitation and its own key.

fherma-runner --name box-cpu --token fhr_...
fherma-runner --name box-gpu-0 --token fhr_...
fherma-runner --name box-gpu-1 --token fhr_...

--name is what keeps them apart on the machine as well as on the platform: the key and the jobs both live under it.

~/.fherma/runners/<name>/
├── key       this runner's own, 0600
└── jobs/     what it is working on

Nothing is shared between them, which is the point — one key answering to two names would be one runner behaving very strangely, and a jobs directory two of them both tidy up is a directory that gets tidied up under one of them.

They do share the machine, so give each one limits it can actually have: --memory and --cpus are per runner and docker will not divide what you promise twice.

What it needs

An interpreter and docker. There are no dependencies: everything here ships with Python, and docker is called as a command.

What it declares

Hardware, and only hardware — cores, memory, architecture, CPU features, an accelerator and its driver. It does not declare libraries or images, because the environment a job runs in is not a property of a machine: it arrives with the job, pinned to a digest.

fherma-runner --describe

prints what it would report, and stops.

What it does with a job

~/.fherma/runners/<name>/jobs/<id>/
├── bundle/     the specification's bundle, unpacked
├── solution/   the repository at the commit, and its build
└── points/
    └── p000/   one point at a time, deleted once reported

Per point: the bundle makes the cases, the solution answers them, the bundle judges the answers. Three containers, and the solution is never given the directory holding the expected answers.

Options

--api           where the API is, not the website      FHERMA_API
--name          what to call this runner               FHERMA_RUNNER_NAME
--token         an invitation, spent once for a key
--interval      seconds between polls                  3
--once          take one assignment, then stop
--describe      print what this machine is, and stop
--version       what is installed, and on what
--re-register   forget this name's key and join again
--memory        what a solution's container may take   FHERMA_MEMORY
--cpus          how many cores it may use              FHERMA_CPUS

The --speed, --fail-rate, --build-seconds and --point-seconds options belong to the half that is still pretended, and go away with it.

Stopping

The platform stops a runner by refusing its heartbeat. It then drops everything at once and sends nothing: finishing the work and reporting it would be reporting into a result somebody else now owns.

If a runner goes quiet instead, its lease lapses, the run is marked expired and the job goes to another machine.

Licence

Apache-2.0.