pypline-ci 0.1.34

A docker pipeline library

pypline-ci

A docker orchestration tool for container-based delivery pipelines - providing developers, testers, and operators a common automated testing tool for local workstations and delivery pipeline infrastructure.

I'm currently experimenting with docker-in-docker delivery pipelines written in a high-level language hoping to capitalize on object-oriented concepts native to Python. While bash does provide a simple ability to script docker-in-docker pipelines; at a certain scale, creating and maintaining independent scripts across multiple integrated repositories or projects becomes cumbersome and inefficient.

Why Python 2.7 by default?

Currently, Python 2.7 is native to most Docker development workstations(popular Linux distros or MacOS). A major focus of this tooling is minimal dependencies however, this library is compatible with Python 3 as well.

Examples

Build image and run container:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
buildPath = dirPath+'/docker/'
localTag = 'local/foo:latest'

pl = Pipeline()
pl.build_image(buildPath, localTag)
pl.rund(localTag)

Implement testing framework from dedicated testing image:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
volumes = {dirPath: {'bind': '/tmp', 'mode': 'rw'}}
testDir = '/tmp/tests'

pl = Pipeline(dockerRegistry='registry.gitlab.com/christiantragesser/')
pl.runi(image=pipeline.dockerRegistry+'my-test-image:latest',
              name='foo-test', working_dir=testDir,
              volumes=volumes, command='pytest')

Orchestrate application stack for UAT testing then remove all containers if tests are successful:

import os
from pyplineCI import Pipeline

dirPath = os.path.dirname(os.path.realpath(__file__))
cleanUp = []

uat_volume = {dirPath: {'bind': '/tmp', 'mode': 'rw'}}
testDir = '/tmp/tests'
db_env_vars = {'MYSQL_ROOT_PASSWORD': 'root', 'MYSQL_DATABASE': 'foo-db', 'MYSQL_ROOT_HOST': '%'}
app_env_vars = {'DB_HOST': 'mysql-test', 'DB_USER': 'root', 'DB_PASSWORD': 'root', 'DATABASE': 'foo-db'}

pl = Pipeline()
cleanUp.append(pl.rund(image='mysql:5.7', name='mysql-test', environment=db_env_vars))
cleanUp.append(pl.rund(image='local/foo_app', name='foo-app-test', environment=app_env_vars))
pl.runi(image='tutum/curl:latest', name='foo-uat',
        working_dir=testDir, volumes=uat_volume,
        command='./uat.sh foo-app-test:5000')
pl.purge_containers(cleanUp)

Perform CVE scan on a docker image:

MY_IMAGE="alpine:latest"
python3 -c "from pyplineCI import Pipeline; Pipeline().cve_scan(${MY_IMAGE})"

API reference

• Pipeline(network='ci_net', dockerRegistry='library/')

  class pypline-ci.pyplineCI.Pipeline

  • create_network( network ) | Create docker pipeline network.
    parameters:
    • network(str) - Name of pipeline network, default ci_net
  • build_image( path, tag ) | Build docker image.
    parameters:
    • path(str) - Path to the directory containing the Dockerfile.
    • tag(str) - Tag applied to newly built image.
  • pull_image( image ) | Pull an image of the given name, similar to the docker pull command. If no tag is specified, all tags from that repository will be pulled.
    parameters:
    • image(str) - Image name to pull.
  • rund( image, stderr=None, ports=None, volumes=None, name=None, environment=None, network=<obj network>, command=None, detach=True, remove=False ) | Performs pull action on provided image, runs a daemonized container, then returns the container ID.
    parameters:
    • environment(dict or list) - Environment variables to set inside the container.
    • image(str) - The image to update and run.
    • name(str) - The name for this container.
    • ports(dict) - Port bindings to the container. The keys of the dictionary are the ports to bind inside the container, either as an integer or a string in the form port/protocol, where the protocol is either tcp or udp. The values of the dictionary are the corresponding ports to open on the host.
    • volumes(dict) - Configure volumes mounted inside the container.
  • runi( image, command, name=None, volumes=None, working_dir='/root', tty=True, environment=None, stdin_open=True, network=<obj network>, auto_remove=False ) | Performs pull action on provided image, runs an interactive container implementing provided command, then returns container stdout logs and command exit status(zero or non-zero).
    parameters:
    • command(str) - The command to run in the container.
    • environment(dict or list) - Environment variables to set inside the container.
    • image(str) - The image to update and run.
    • name(str) - The name for this container.
    • ports(dict) - Port bindings to the container. The keys of the dictionary are the ports to bind inside the container, either as an integer or a string in the form port/protocol, where the protocol is either tcp or udp. The values of the dictionary are the corresponding ports to open on the host.
    • volumes(dict) - Configure volumes mounted inside the container.
    • working_dir(str) - Path to the working directory.
  • purge_containers( ids ) | Force deletion of container by container ID.
    parameters:
    • ids(list) - List of container IDs to delete.
  • cve_scan( scanImage ) | Perform CVE scan of docker image using CoreOS Clair.
    parameters:
    • scanImage(str) - The image to scan.

Install

Install on docker host:

• From PyPi

$ pip install pypline-ci

or

• Via docker:

$ docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v $PWD:/tmp \
    -w /tmp \
    registry.gitlab.com/christiantragesser/pypline-ci:latest /bin/sh

• For Python 3:

$ docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v $PWD:/tmp \
    -w /tmp \
    registry.gitlab.com/christiantragesser/pypline-ci:3 /bin/sh