Skip to main content

Build container images faster ⚡️

Project description

Build container images faster ⚡️

https://xkcd.com/303/

https://github.com/lexsca/prefab/actions/workflows/checks.yml/badge.svg https://img.shields.io/pypi/v/container-prefab.svg https://img.shields.io/pypi/pyversions/container-prefab.svg https://img.shields.io/github/license/lexsca/prefab.svg https://img.shields.io/badge/code%20style-black-000000.svg

Prefab is a Python-based container image build tool that uses deterministic remote caching to reduce build times. Unlike BuildKit and the Docker CLI, which use build layer caching, Prefab uses whole image caching based on a digest of the Dockerfile in combination with digests of specified files and directory trees. This allows Prefab to check for and pull cached images before resorting to building a new image.

Quickstart

Look at the example directory to see how to build an example app with Prefab.

Installation and usage

Prefab can be installed and run in three different ways:

  1. Local Python package

  2. Docker outside of Docker (DooD) container

  3. Docker in Docker (DinD) container

Use whichever mode works best for the use-case(s) at hand. Each supports the same CLI arguments:

CLI arguments

usage: prefab [-h] [--config PATH] [--dry-run] [--force] [--monochrome]
              [--push TARGET_NAME [TARGET_NAME ...]] [--push-all] --repo URI
              --target TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...]

Build container images faster ⚡️

optional arguments:
  -h, --help            show this help message and exit
  --config PATH, -c PATH
                        Target build config file to use (default: prefab.yml)
  --dry-run             Show how targets would be built (implies --force)
  --force               Force target(s) to be rebuilt
  --monochrome, -m      Don't colorize log messages
  --push TARGET_NAME [TARGET_NAME ...], -p TARGET_NAME [TARGET_NAME ...]
                        Image target(s) to push to repo after building
  --push-all            Push all image targets to repo after building
  --repo URI, -r URI    Image repo to use (e.g. lexsca/prefab)
  --target TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...], -t TARGET_NAME[:TAG] [TARGET_NAME[:TAG] ...]
                        Image target(s) to build with optional custom image tag

Local Python package

To install Prefab as a local Python package:

pip install container-prefab

To run Prefab as a local Python package to build an push a build target:

prefab --repo repo.tld/org/project --push --target name

NOTE: Container images now hosted on Docker Hub

The container images below used to be hosted by Quay and are now hosted by Docker Hub This decision was not taken lightly. Sadly, Quay has not proven to be a reliable service. The final straw was when RedHat acquired them and broke authentication.

Docker outside of Docker (DooD)

To get the Prefab Docker outside of Docker (DooD) image:

docker pull lexsca/prefab:dood

To run the Prefab Docker outside of Docker image to build an push a build target:

docker run --rm -it -v $(/bin/pwd):/build -w /build \
    -e REGISTRY_AUTH=$(jq -c . ~/.docker/config.json | base64) \
    -v /var/run/docker.sock:/docker.sock \
    lexsca/prefab:dood --repo repo.tld/org/project \
    --push --target name

Docker in Docker (DinD)

To get the Prefab Docker in Docker (DinD) image:

docker pull lexsca/prefab:dind

To run the Prefab Docker in Docker image to build an push a build target:

docker run --rm -it -v $(/bin/pwd):/build -w /build --privileged \
    -e REGISTRY_AUTH=$(jq -c . ~/.docker/config.json | base64) \
    lexsca/prefab:dind --repo repo.tld/org/project \
    --push --target name

Configuration

Prefab uses a YAML configuration file to determine which container images to build for a given target and in which order. This configuration below is taken from the example directory in this repo.

The prefab.yml file has two build targets, each with a Dockerfile. The app target has a dependency on the packages target, so it’s built or pulled first, before building the app target. This is a simple example, but the dependency graph can be arbitrarily deep or wide for complex build targets.

prefab.yml

targets:

  app:
    dockerfile: Dockerfile.app
    depends_on:
      - packages
    watch_files:
      - app.py

  packages:
    dockerfile: Dockerfile.packages

When building a container image, Prefab populates build arguments for each build target depndency, uppercased by convention, and prefixed with PREFAB_ to avoid conflicts with other build arguments.

Dockerfile.app

ARG PREFAB_PACKAGES

FROM $PREFAB_PACKAGES as packages

Contributing

Bug reports are welcome. Pull requests even more so.

Before making any changes, first ensure the development environment is functional and the extant linting and tests are passing. To start a development environment, clone or fork this source repo and follow the instructions below.

Alternatively, it’s fine to create a virtual environment an install packages from requirements.txt and requirements-dev.txt files. The Python version should be 3.7 or later.

Prerequisites

  1. POSIX Shell (e.g. bash)

  2. Docker

  3. GNU Make

Create environment

To create a development runtime environment:

$ make bootstrap

The above will create a minimal environment that will allow Prefab to build its development environment image. This image can be used to run linting and tests:

$ docker images lexsca/prefab:dev
REPOSITORY      TAG                 IMAGE ID            CREATED              SIZE
lexsca/prefab   dev                 ddee1cafb775        About a minute ago   429MB

Use environment

Once created, the development image can used via:

$ make shell
docker run --rm -it -v /Users/lexsca/git/prefab:/prefab -w /prefab \
        -v /var/run/docker.sock:/docker.sock -e PYTHONPATH=/prefab/lib \
        --entrypoint /bin/bash lexsca/prefab:dev --login -o vi
3053ae861610:/prefab# make test

This will mount the docker socket and current working directory in an environment where tests can be run, dependencies built, or a debugger invoked to aid in iterating.

The make test command should pass before attempting to submit any code changes.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

container-prefab-21.10.172102.tar.gz (29.8 kB view details)

Uploaded Source

Built Distribution

container_prefab-21.10.172102-py3-none-any.whl (17.9 kB view details)

Uploaded Python 3

File details

Details for the file container-prefab-21.10.172102.tar.gz.

File metadata

  • Download URL: container-prefab-21.10.172102.tar.gz
  • Upload date:
  • Size: 29.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for container-prefab-21.10.172102.tar.gz
Algorithm Hash digest
SHA256 5c1346f829d64c3a919bc3a11764ed563d048ef52cf0a445b43cd18068d064fb
MD5 2d83d6cdc62252085f566a2ac1c23ff5
BLAKE2b-256 27671eaa0a9ae54ab7f65427f0c32caae17785a55f5120bc619cc22dbe95ccbf

See more details on using hashes here.

File details

Details for the file container_prefab-21.10.172102-py3-none-any.whl.

File metadata

  • Download URL: container_prefab-21.10.172102-py3-none-any.whl
  • Upload date:
  • Size: 17.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for container_prefab-21.10.172102-py3-none-any.whl
Algorithm Hash digest
SHA256 b199d868fb3433911faf4e817be50dc27dbffcb5c117c943cc3c362965fa44e8
MD5 843e6c075fb00db776f923ca0251ce10
BLAKE2b-256 1154c09918fe5ecf54b3799b9f3e2ecb8a5e84422675d34ac7b733c3c5a1e1cb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page