container-magic 4.4.0

A tool for rapidly creating containerised development environments

container-magic

Define your container workflow in a single YAML file. Container-magic generates a Dockerfile, build script, and run script that work with Docker or Podman - container-magic is not a dependency of your final product.

How It Works

You write a cm.yaml. Container-magic generates a Dockerfile, build.sh, and run.sh from it. These generated files are committed to your repository so that anyone can build and run the project with Docker or Podman.

For development, cm build and cm run read the config directly and handle workspace mounting, user identity mapping, and runtime features automatically.

Quick Start

pip install container-magic
cm init python:3.11 my-project
cd my-project
cm build
cm run python --version

A minimal cm.yaml:

names:
  image: my-project
  workspace: workspace
  user: nonroot

stages:
  base:
    from: python:3.11-slim
    steps:
      - apt-get:
          install:
            - git
            - build-essential
      - pip:
          install:
            - numpy
            - pandas

  development:
    from: base

  production:
    from: base

Build and run the production image:

./build.sh
./run.sh python workspace/train.py

Key Features

• Development and production from one config - live-mounted workspace in dev, baked-in code in prod
• Automatic user handling - host user identity in dev, dedicated user in prod, no manual setup
• GPU, display, and audio - NVIDIA GPU passthrough, X11/Wayland forwarding, PulseAudio/PipeWire
• Custom commands - define once, use in both dev and prod with port publishing and environment variables
• Multi-stage builds - share steps between stages, automatic virtual environments for pip
• Transparent execution - run commands from anywhere in your repo with automatic path translation
• Data volumes - shorthand for sibling folders (outputs, cache) that persist across runs without entering the image
• AWS credential forwarding - mount host AWS config into the container
• Cached assets - download models and datasets once, reuse across builds

Documentation

Full documentation is available at markhedleyjones.com/container-magic.

Page	Contents
Getting Started	Installation, first project, workflow
Configuration	Full YAML reference - names, runtime, stages, commands
Build Steps	Package managers, custom commands, layer caching
Cached Assets	Asset downloading, caching, and cache management
User Handling	Dev vs prod users, copy ownership, permissions
Troubleshooting	Common issues and solutions

Contributing

Contributions and feedback welcome! Open an issue or pull request on GitHub.