trycicle 0.6.1

Try CI jobs locally

Trycicle

Try CI jobs locally, including services.

Installation

pipx install trycicle

Tab completion

Trycicle supports tab completion for Bash, Zsh and Fish.

Bash

Add this to your ~/.bashrc:

eval "$(_TRYCICLE_COMPLETE=bash_source trycicle)"

Zsh

Add this to your ~/.zshrc:

eval "$(_TRYCICLE_COMPLETE=zsh_source trycicle)"

Fish

Save the completion script to your completions directory:

_TRYCICLE_COMPLETE=fish_source trycicle > ~/.config/fish/completions/trycicle.fish

Usage

Trycicle is a command line tool that allows you to run CI jobs locally. It expects the name of the CI job to run as the first argument. It will look for a .gitlab-ci.yml file in the current directory, unless the --file option is used to specify a different file. It assumes the directory containing the .gitlab-ci.yml is the source directory, this can be overridden with the --workdir option.

Examples

Basic usage

Given the following .gitlab-ci.yml file:

image: busybox:latest

variables:
  NAME: world

test:
  script:
    - echo "Hello, $NAME!"

You can run the test job with trycicle test:

$ trycicle test
INFO:trycicle.run:Starting job test (busybox:latest)
...
Hello, world!

Trycicle will tell you which jobs (and services) it runs. Script commands are also echoed. To get more detailed output, use --verbose.

Variables

Trycicle defines many of the commonly used GitLab CI predefined variables. If it notices your job depends on a variable that is not set, it will print a warning:

image: busybox:latest

test:
  script:
    - echo "CI_JOB_NAME=$CI_JOB_NAME"
    - echo "GREETING=$GREETING"
    - echo "MISSING=$MISSING"
  variables:
    GREETING: "Hello, $USER!"

$ trycicle test
INFO:trycicle.run:Starting job test (busybox:latest)
WARNING:trycicle.variables:Undefined variable 'USER'
...
CI_JOB_NAME=test
...
GREETING=Hello, $USER!
...
MISSING=

As you can see, Trycicle can not detect all cases where a variable is missing. This is most noticeable when undefined variables are used in scripts, where the shell will expand them to an empty string. (To help with this, you can use set -u in your scripts to make the shell fail when an undefined variable is used. Of course, this works on actual GitLab CI as well.)

To pass an extra variable to the job, use the --env (or -e) option:

$ trycicle test --env USER=Trycicle --env MISSING=found
...
GREETING=Hello, Trycicle!
...
MISSING=found

When a variable is defined in your local environment, use --env with just the name to pass it to the job:

$ trycicle test --env USER
...
GREETING=Hello, paul!

Services

Trycicle can run services for your job:

test:
  image: busybox:latest
  services:
    - nginx:latest
  script:
    - wget -qO- http://nginx/

$ trycicle test
INFO:trycicle.run:Starting service nginx (nginx:latest)
INFO:trycicle.run:Starting job test (busybox:latest)
...
<h1>Welcome to nginx!</h1>

If a service container exits while the job is still running, Trycicle will print a warning:

test:
  image: busybox:latest
  services:
    - name: busybox:latest
      command: [sh, -c, "sleep 1; exit 1"]
  script:
    - sleep 2
    - echo "Done"

$ trycicle test
INFO:trycicle.run:Starting service busybox (busybox:latest)
INFO:trycicle.run:Starting job test (busybox:latest)
...
WARNING:trycicle.run:Service busybox died with exit code 1
...
Done

To display the logs of the service containers, use --service-logs. Each line of output is prefixed with the name of the service, or job:

test:
  image: busybox:latest
  services:
    - name: postgres:latest
      command: postgres -c log_line_prefix=
  variables:
    POSTGRES_PASSWORD: postgres
  script:
    - while ! nc -z postgres 5432; do sleep 1; done
    - echo "Done"

$ trycicle test --service-logs
INFO:trycicle.run:Starting service postgres (postgres:latest)
INFO:trycicle.run:Starting job test (busybox:latest)
...
postgres | running bootstrap script ... ok
...
postgres | LOG:  database system is ready to accept connections
...
job | Done

Docker-in-Docker

Trycicle tries to detect when your job is using Docker-in-Docker and will run with the --privileged option. It will also use a volume to share the Docker certificates between job and service.

test:
  image: python:3.11
  services:
    - docker:20-dind
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: /certs
    DOCKER_TLS_VERIFY: 1
    DOCKER_CERT_PATH: "${DOCKER_TLS_CERTDIR}/client"
  script:
    - pip install docker~=7.0.0
    - |
      python -c 'import docker
      client = docker.from_env()
      output = client.containers.run("busybox:latest", ["echo", "Hello", "DinD!"], remove=True)
      print(output.decode())
      '

$ trycicle test
INFO:trycicle.run:Starting service docker (docker:20-dind)
INFO:trycicle.run:Starting job test (python:3.11)
...
Hello DinD!

Clean copy

By default, Trycicle runs the job directly in your source directory. This way you can edit files and run the job again without having to commit and push. This also means that the job has access to files that would not normally be committed, like node_modules/ or a .env file with settings specific to your local environment.

It also means that the job can create or modify files in your source directory. This is great to inspect the output of the job, but it can also lead to unexpected changes in your source directory.

To avoid this, Trycicle can use Git to create a clean copy of your source directory and run the job there. Any changes not committed will not be visible to the job. Your changes do not need to be pushed. Changes to the .gitlab-ci.yml are used.

test:
  image: buildpack-deps:stable-scm
  script:
    - git remote -v
    - cat hello.txt
    - git status

$ git init .
$ echo 'Hello, world!' > hello.txt
$ git add hello.txt
$ git commit -m 'Initial commit'
$ echo 'Another file' > untracked.txt
$ echo 'An uncommitted edit' >> hello.txt
$ trycicle test --clean
...
+ git remote -v
origin	/repo (fetch)
origin	/repo (push)
+ cat hello.txt
Hello, world!
+ git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

As you can see, only the committed changes are visible to the job. Trycicle adds your source directory as a remote to the clean copy, this makes it possible to use git fetch in the job. Pushing does not work, your working directory is mounted read-only.

Caching

Trycicle has rudimentary support for caching. If a job defines a cache key, Trycicle will keep matching files between runs.

image: busybox:latest

cache:
  key: example
  paths:
    - "*.txt"

one:
  script:
    - echo 'Hello, world!' > hello.txt

two:
  script:
    - cat hello.txt

Running the first job will create a cache with the hello.txt file:

$ trycicle one
INFO:trycicle.cache:Using cache 'cache-readme-example'
...

The second job will be able to read the file from the cache:

$ trycicle two
INFO:trycicle.cache:Using cache 'cache-readme-example'
...
Hello, world!

Includes

Trycicle supports includes and components, and does its best to match GitLab CI behavior.

include:
  - local: extra.gitlab-ci.yml
  - template: Jobs/SAST.gitlab-ci.yml
  - component: gitlab.com/components/secret-detection/secret-detection@1.1.1

Jobs from included configuration or components can be run just like any other job:

$ printf 'extra:\n  script:\n    - echo "Hello, included!"\n' > extra.gitlab-ci.yml
$ trycicle
...
Available jobs:
  - extra
  - sast
  - gitlab-advanced-sast
...
  - secret_detection

$ echo '{}' > extra.gitlab-ci.yml
$ trycicle secret_detection
INFO:trycicle.run:Starting job secret_detection (registry.gitlab.com/security-products/secrets:6)
...
$ ls gl-secret-detection-report.json
gl-secret-detection-report.json

Included repositories are cloned using SSH. To use private component projects, make sure the $GITLAB_TOKEN environment variable contains a valid Personal Access Token.

Debugging

Logs

Trycicle does not immediately remove the containers it creates, so you can inspect them after the job has finished. The containers are labeled with the job they were part of, so you can use a command like docker ps -a --filter label=trycicle.job=test to list them. This can also be combined with docker logs, for example to get the logs of postgresql service in the most recent run:

docker logs $(docker ps -q --latest --filter label=trycicle.service=postgres)

The labels Trycicle uses are:

Label	Value	Notes
trycicle	Always true	Can be used to list all containers created by Trycicle: docker ps --all --filter label=trycicle
trycicle.job	Name of the job
trycicle.service	Name of the service	Only set for services
trycicle.workdir	Full path to the working directory	Usually the directory containing the .gitlab-ci.yml

Interactive debugging

Trycicle can run a job in interactive mode, which means you can execute commands in the container after the job has finished. This is useful to inspect the state of the container, check environment variables, or find out what that missing package is called.

The debug shell will start automatically when the job fails, you can also add a debugger command to the script to start it manually. To enter the debug shell before the job starts, use the --debug=immediate option.