modelcar-maker

Downloads Hugging Face models and packages them as minimal OCI container images with one file per layer. Builds on top of olot with a full publishing workflow. Designed for use with KServe as Modelcar images.

Install

Requires Python 3.11+ and skopeo available in $PATH

pip install modelcar-maker

Usage

modelcar-maker [OPTIONS] [MODEL]

Image tag format: {registry}/{repository}:{specified tag or normalized-model}-modelcar

With --output json, stdout contains exactly one JSON document with the overall status, operation duration, and a result for each attempted model. Per-model results include status (success, skipped, or failed), durations, errors, and available payload, build, destination, and push metadata. The download object's payload_files and payload_bytes are totals in the resulting local payload, not counts of bytes transferred. Logs and diagnostics are written to stderr. A failed model makes the command exit nonzero; an operation with both successes and failures has status partial.

After download, image construction excludes files with any dot-prefixed path component, plus files named Containerfile, original, and consolidated.safetensors. Other downloaded files, including PyTorch files, are not excluded by this filter. --quiet affects text progress and success messages only; JSON output remains the final stdout document. --fail-fast stops attempting further models after the first model-specific failure.

When no tag is specified, the model repo ID is normalized for use as a tag: slashes become --, dots become _, and the string is lowercased before appending -modelcar. For example, meta-llama/Llama-3.2-3B-Instruct produces tag meta-llama--llama_3-2-3b-instruct-modelcar.

Examples:

# Build and push a model
modelcar-maker meta-llama/Llama-3.2-3B-Instruct

# Build locally without pushing
modelcar-maker BAAI/bge-large-en-v1.5 --no-push

# Build with cleanup
modelcar-maker mistralai/Mistral-7B --image-clean-up --model-clean-up

Configuration

Settings are resolved in layer order (later layers override earlier ones):

1. Built-in defaults (defaults.toml useful as a reference)
2. /usr/share/modelcar-maker/config.toml
3. /etc/modelcar-maker/config.toml
4. ~/.config/modelcar-maker/config.toml
5. ./config.toml (current working directory)
6. The config in the path described by the MODELCAR_MAKER_CONFIG environment variable
7. Environment variables with prefix MODELCAR_MAKER_
8. CLI arguments override all of the above

Any setting can be overridden via environment variable by uppercasing the section and key and joining with double underscores. For example:

Models may have additional configuration applied to them by using the model repository ID in a subkey under the models section of the config. This allows mapping tags and file lists to specific models via config.

Example config.toml:

[image]
registry = "quay.io"
repository = "myorg/models"

[models]
default = [
  "meta-llama/Llama-3.2-3B-Instruct",
  "BAAI/bge-large-en-v1.5",
  "unsloth/Qwen3.5-9B-MTP-GGUF",
]
"unsloth/Qwen3.5-9B-MTP-GGUF".files = ["README.md", "Qwen3.5-9B-Q4_K_S.gguf"]
"unsloth/Qwen3.5-9B-MTP-GGUF".tag = "unsloth--qwen3_5-9b-q4_k_s-modelcar"

Setting default models allows running modelcar-maker with no arguments to build all listed models.

Running as a Container

The tool is available as a container image at ghcr.io/jharmison-redhat/modelcar-maker:latest (versioned tags also available).

podman run --rm ghcr.io/jharmison-redhat/modelcar-maker:latest --help

For authenticated pushes, mount a credential file. For gated downloads, provide the HF_TOKEN environment variable:

podman run --rm \
  -v "$HOME/.docker/config.json:/config.json" \
  -e HF_TOKEN="$HF_TOKEN" \
  ghcr.io/jharmison-redhat/modelcar-maker:latest \
  --authfile /config.json \
  meta-llama/Llama-3.2-3B-Instruct

To persist downloaded model files across runs, mount a volume at /modelcar-maker:

podman run --rm \
  -v modelcar-cache:/modelcar-maker \
  -v "$HOME/.docker/config.json:/config.json" \
  -e HF_TOKEN="$HF_TOKEN" \
  ghcr.io/jharmison-redhat/modelcar-maker:latest \
  --authfile /config.json \
  --skip-if-exists \
  meta-llama/Llama-3.2-3B-Instruct

docker may work but is untested.

Running in Kubernetes/OpenShift

A helm chart is published at oci://ghcr.io/jharmison-redhat/modelcar-maker-chart (with versioned tags).

You can view the full default values list with the following command:

helm show values oci://ghcr.io/jharmison-redhat/modelcar-maker-chart

A minimal values.yaml to publish a few models to an ImageStream in an OpenShift cluster might look like this (but probably with a real HF_TOKEN specified):

---
huggingFace:
  token: hf_asdfasdf
models:
  - repo: TinyLlama/TinyLlama-1.1B-Chat-v1.0
  - repo: unsloth/Qwen3.5-9B-MTP-GGUF
    files:
      - README.md
      - Qwen3.5-9B-Q4_K_S.gguf
    tag: unsloth--qwen3_5-9b-q4_k_s-modelcar
  - repo: unsloth/Nemotron-3-Nano-30B-A3B-GGUF
    files:
      - README.md
      - Nemotron-3-Nano-30B-A3B-UD-Q8_K_XL.gguf
    tag: unsloth--nemotron-3-nano-30b-a3b-ud-q8_k_xl-modelcar

Which, if that file was named values.yaml in your local directory, you could install with a command like this:

helm_args=(
    upgrade --install                                   # this lets you reuse the same command to upgrade
                                                        # or change values later

    -n modelcars --create-namespace                     # pick whichever namespace you like

    modelcar-maker                                      # the release-name would allow you to install different
                                                        # copies of the chart,
                                                        # for all the reasons you might want to do that

    oci://ghcr.io/jharmison-redhat/modelcar-maker-chart # You can specify a version tag, too, if you like

    -f values.yaml                                      # Keep your values updated over time to rerun
)
helm "${helm_args[@]}"

For non-OpenShift Kubernetes, you'll want to disable the ImageStream and specify a destination registry and push secret. A minimal configuration for that looks like this (note that you still need to specify models):

---
modelcar:
  image:
    registry: quay.io
    repository: youruser/yourrepo
  imageStream:
    create: false
  dockerConfigJson: |
    {
      "auths": {
        "quay.io": {
          "auth": "ZmFrZXVzZXI6ZmFrZXBhc3N3b3Jk"
        }
      }
    }

The dockerConfigJson section here should be the entire pull secret necessary for building your Modelcar image, including pulling the base image (if you override the publicly pullable default) and pushing the built image. There are lots of ways to make these, including by hand, using podman login with credentials from your registry and copying the file it creates, etc. It doesn't require any special handling and is passed directly to skopeo as ~/.docker/config.json which it always falls back to using.

Credentials and workspace behavior

For an explicit registry credential, set modelcar.dockerConfigJson inline in the values supplied to Helm. The chart creates the current generated Secret from that value and mounts it as the Docker config used by skopeo; supply a complete config that can pull the selected base image and push the destination image.

When modelcar.dockerConfigJson is empty, service-account credentials are the only supplied credentials. The main container runs modelcar-maker --service-account-auth, which constructs a Docker config from the live projected service-account token before every registry operation (the credential is always fresh, so builds outliving the token TTL still push). It writes it only to a memory-backed ephemeral volume, never to the workspace or PVC. They can work with the OpenShift internal registry or an unauthenticated writable registry; other authentication failures appear in the Job logs.

The workspace can be an emptyDir or PVC. A PVC retains model and image workspace data when selected, but never the service-account Docker credentials.

Troubleshooting

• Hugging Face access: provide HF_TOKEN for gated models or when your access/rate limit requires it.
• Registry authentication: verify that modelcar.dockerConfigJson covers both base-image pulls and destination pushes. For service-account mode, use the OpenShift internal registry or an unauthenticated writable registry.
• CA trust: enable the cluster CA option or provide the required custom CA in certificateAuthorities.custom.
• Workspace space: choose a workspace size that accommodates the downloaded model, staging/reconciliation, and OCI layout; increase a PVC size or configure workspace.emptyDir limits accordingly.
• Retained Job name conflicts: Jobs are retained intentionally. Changing an execution input produces a new name; otherwise remove the retained Job deliberately before reusing the same release and inputs.

License

ISC. See LICENSE.