pypline-ci

A docker orchestration tool for container-based delivery pipelines - providing developers, testers, and operators a common automated testing tool for local workstations and delivery pipeline infrastructure.

I'm currently experimenting with docker-in-docker delivery pipelines written in a high-level language hoping to capitalize on object-oriented concepts native to Python. While bash does provide a simple ability to script docker-in-docker pipelines; at a certain scale, creating and maintaining independent scripts across multiple integrated repositories or projects becomes cumbersome and inefficient.

Examples

Build image and run container:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
buildPath = dirPath+'/docker/'
localTag = 'local/foo:latest'

pl = Pipeline()
pl.build_image(buildPath, localTag)
pl.rund(localTag)

Implement testing framework from dedicated testing image:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
volumes = {dirPath: {'bind': '/tmp', 'mode': 'rw'}}
testDir = '/tmp/tests'

pl = Pipeline(dockerRegistry='registry.gitlab.com/christiantragesser/')
pl.runi(image=pipeline.dockerRegistry+'my-test-image:latest',
              name='foo-test', working_dir=testDir,
              volumes=volumes, command='pytest')

Orchestrate application stack for UAT testing then remove all containers if tests are successful:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
cleanUp = []

uat_volume = {dirPath: {'bind': '/tmp', 'mode': 'rw'}}
testDir = '/tmp/tests'
db_env_vars = {'MYSQL_ROOT_PASSWORD': 'root', 'MYSQL_DATABASE': 'foo-db', 'MYSQL_ROOT_HOST': '%'}
app_env_vars = {'DB_HOST': 'mysql-test', 'DB_USER': 'root', 'DB_PASSWORD': 'root', 'DATABASE': 'foo-db'}

pl = Pipeline()
cleanUp.append(pl.rund(image='mysql:5.7', name='mysql-test', environment=db_env_vars))
cleanUp.append(pl.rund(image='local/foo_app', name='foo-app-test', environment=app_env_vars))
pl.runi(image='tutum/curl:latest', name='foo-uat',
        working_dir=testDir, volumes=uat_volume,
        command='./uat.sh foo-app-test:5000')
pl.purge_containers(cleanUp)

Perform CVE scan on a docker image:

MY_IMAGE="alpine:latest"
python3 -c "from pyplineCI import Pipeline; Pipeline().cve_scan(${MY_IMAGE})"

API reference

• Pipeline(network='ci_net', dockerRegistry='library/')

  class pypline-ci.pyplineCI.Pipeline

  • create_network( network ) | Create docker pipeline network.
    parameters:
    • network(str) - Name of pipeline network, default ci_net
  • build_image( path, tag ) | Build docker image.
    parameters:
    • path(str) - Path to the directory containing the Dockerfile.
    • tag(str) - Tag applied to newly built image.
  • pull_image( image ) | Pull an image of the given name, similar to the docker pull command. If no tag is specified, all tags from that repository will be pulled.
    parameters:
    • image(str) - Image name to pull.
  • rund( image, stderr=None, ports=None, volumes=None, name=None, environment=None, network=<obj network>, command=None, detach=True, remove=False ) | Performs pull action on provided image, runs a daemonized container, then returns the container ID.
    parameters:
    • environment(dict or list) - Environment variables to set inside the container.
    • image(str) - The image to update and run.
    • name(str) - The name for this container.
    • ports(dict) - Port bindings to the container. The keys of the dictionary are the ports to bind inside the container, either as an integer or a string in the form port/protocol, where the protocol is either tcp or udp. The values of the dictionary are the corresponding ports to open on the host.
    • volumes(dict) - Configure volumes mounted inside the container.
  • runi( image, command, name=None, volumes=None, working_dir='/root', tty=True, environment=None, stdin_open=True, network=<obj network>, auto_remove=False ) | Performs pull action on provided image, runs an interactive container implementing provided command, then returns container stdout logs and command exit status(zero or non-zero).
    parameters:
    • command(str) - The command to run in the container.
    • environment(dict or list) - Environment variables to set inside the container.
    • image(str) - The image to update and run.
    • name(str) - The name for this container.
    • ports(dict) - Port bindings to the container. The keys of the dictionary are the ports to bind inside the container, either as an integer or a string in the form port/protocol, where the protocol is either tcp or udp. The values of the dictionary are the corresponding ports to open on the host.
    • volumes(dict) - Configure volumes mounted inside the container.
    • working_dir(str) - Path to the working directory.
  • purge_containers( ids ) | Force deletion of container by container ID.
    parameters:
    • ids(list) - List of container IDs to delete.
  • cve_scan( scanImage ) | Perform CVE scan of docker image using CoreOS Clair.
    parameters:
    • scanImage(str) - The image to scan.

Install

Python 3.6 or later

Install on docker host:

• From PyPi

$ pip install pypline-ci

or

• Via docker:

$ docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v $PWD:/tmp \
    -w /tmp \
    registry.gitlab.com/christiantragesser/pypline-ci /bin/sh