gitlab-runner-api 1.0.4

Unofficial implementation of the GitLab runner client for making customised CI runners

An unofficial Python implementation of the API for creating customised GitLab CI runners.

This package provides the basic functionality for registering a Runner. This object can then be used to request a Job and communicate the log, status and artifacts back to GitLab. No functionality is provided to execute the payloads defined in the .gitlab-ci.yml.

This package was originally developed to support LHCb’s Analysis Productions by providing a CI runner that could submit jobs to LHCbDIRAC. More details can be found in TODO.

Registering a Runner

The simplest way to register a new Runner is with the included command line tool:

$ register-runner --help
usage: register-runner [-h] [--locked] [--maximum-timeout MAXIMUM_TIMEOUT] api_url token output_fn

positional arguments:
api_url
token
output_fn

optional arguments:
-h, --help            show this help message and exit
--locked              Lock the runner the to the current project
--maximum-timeout MAXIMUM_TIMEOUT
                        Maximum timeout set when this Runner will handle the job (in seconds)

For example

$ register-runner "https://gitlab.cern.ch/" "MY_REGISTRATION_TOKEN" "my-runner-data.json " --locked
INFO:gitlab_runner_api:gitlab.cern.ch: Successfully registered runner 6602 (abcdefghij)
INFO:gitlab_runner_api:gitlab.cern.ch: Successfully initialised runner 6602

where arguments can be found by navigating to the “CI/CD” page of the desired repository’s settings.

Getting jobs

After a runner has been registered it can be loaded from the .json file and used to request jobs:

from gitlab_runner_api import Runner
runner = Runner.load("my-runner-data.json")
runner.check_auth()
if job := runner.request_job():
    print("Received a new job, starting executor")
    my_job_executor(job)
else:
    print("No jobs are currently available")

Executing jobs

A minimal CI executor might run as follows:

from gitlab_runner_api import failure_reasons

job.log += f"Starting job with id {job.id} for branch {job.ref}\n"
do_clone_and_checkout(job.repo_url, job.commit_sha)
success = run_tests(job)
if success:
    job.log += "All tests ran successfully\n"
    job.set_success()
else:
    # ANSI formatting codes can be used to enhance the CI logs
    job.log += "\u001b[31mJob failed!!!\u001b[0m\n"
    job.set_failed(failure_reasons.ScriptFailure())

See the reference Job documentation for the full list of available properties.

Persisting jobs

For long running jobs it may be desirable to persist the job object between calls. This can be done using a similar interface to the pickle module in the Python standard library:

job_data = job.dumps()

from gitlab_runner_api import Job
job = Job.loads(job_data)

Note: The job log is included in the persisted data therefore the Job object cannot be persisted once and loaded multiple times without loosing the log messages.