cibuildmp

Build MicroPython native C extensions for every target they support, from one declarative config — on CI and on your own machine. cibuildwheel, for MicroPython.

Covers natmod (dynamically loadable native .mpy modules, built against py/dynruntime.mk) and usermod (USER_C_MODULES, compiled straight into a port's own firmware build) C extensions.

Why

MicroPython gives a native C extension two standard, unrelated build paths:

• natmod — natmod/Makefile includes MicroPython's own py/dynruntime.mk, parameterised by ARCH=. Produces a runtime-loadable .mpy per target architecture. See MicroPython's own examples/natmod/.
• usermod — usermod/micropython.cmake + usermod/micropython.mk, pointed at via USER_C_MODULES= on a port's own build. Compiled straight into the firmware image. See MicroPython's docs/develop/cmodules.rst.

ballistics-lab/micropython-bclibc, o-murphy/micropython-wasm3 and o-murphy/a7p each already follow that layout — that part was never the problem. What diverged was the CI around it: each repo's own GitHub Actions workflow was hand-copied into the next and then evolved independently, so the same ~10-architecture build matrix and the same toolchain-install steps ended up as three separate, slowly drifting copies. cibuildmp is the shared home for the parts that are genuinely identical across all three, resolving each target's cross-toolchain itself instead of leaving it hand-written per repo, the same way locally and on CI.

Quick start

Install it from PyPI and point it at a module with a cibuildmp.toml:

$ uv tool install cibuildmp
$ cibuildmp --build "mpy6.3-*" --dry-run
cibuildmp: 10 target(s) against MicroPython v1.29.0
  [ 1/10] mpy6.3-v1.29.0-x86           make -C natmod ARCH=x86 dist
  [ 2/10] mpy6.3-v1.29.0-x64           make -C natmod ARCH=x64 dist
  ...

An unconfigured build matches nothing at all, from any platform — see Identifiers and selectors below for the full identifier list and glob syntax. Drop --dry-run and it builds for real: each target lands in its own output-dir/<identifier>/ directory (mpyhouse/mpy6.3-x64/, …), with a package.json mip can install from once version is set.

cibuildmp needs a reachable Docker daemon on whatever host runs it. It never builds an image itself — it pulls pre-built, pinned images (ghcr.io/ballistics-lab/<target>) and launches sibling containers, one per target, the same way cibuildwheel's own container runtime does. That covers natmod (a single docker/natmod.Dockerfile for all ten arches) and every usermod port, esp32 included — only the ESP-IDF git clone itself stays on the host (source, not a binary, the same reasoning mpy_dir mounts straight in everywhere else); installing ESP-IDF's own tools and building both run inside esp_idf_base. There is no "run cibuildmp itself inside Docker" story any more — a previous root Dockerfile offered that and was deleted once usermod needed to launch sibling containers of its own (Docker-in-Docker was ruled out); uv tool install cibuildmp/pip install cibuildmp directly is the one supported way to run it outside CI.

A non-native target (anything other than your own machine's architecture) also needs an emulator registered: docker/setup-qemu-action@v4 on CI, or once per machine locally, docker run --privileged --rm tonistiigi/binfmt --install all. cibuildmp names the missing emulator up front rather than letting the build fail with exec format error.

On CI, use the action instead of installing the CLI yourself — it already runs on a bare runner with the runner's own Docker daemon reachable:

- uses: ballistics-lab/cibuildmp@v0.3.0
  with:
    build: "mpy6.3-* v1.29.0-manylinux_2_28_x86_64"

See examples/template for a minimal natmod module and its cibuildmp.toml, and examples/wasm2mpy for one whose native source is WebAssembly, compiled through wasm2c — the natmod contract doesn't care what produced the C. cibuildmp --help lists every flag; CIBMP_* environment variables and a [tool.cibuildmp] table in pyproject.toml both work as config overrides too.

Identifiers and selectors

Every buildable thing — one natmod arch, one usermod port/board/arch cell — has a real, stable identifier, read straight from resources/build-platforms.toml (never guessed or reconstructed from a format string). build/skip (config, CIBMP_BUILD/CIBMP_SKIP env vars, or --build/--skip on the CLI) are space-separated glob patterns matched against these identifiers, skip applied after build; an [override."<glob>"] table applies option overrides to whichever identifiers match its own glob. There's no other selection mechanism — no per-platform table, no --platform/--only flag, no auto/native/all keyword vocabulary. An unconfigured build selects nothing, from any platform.

Identifier shapes, one per platform:

The shape genuinely differs per usermod port — unix/windows/ webassembly carry no port name at all in the identifier, only qemu/ esp32/rp2 do. --print-build-identifiers --json against a broad build glob is the fastest way to see the real list for yourself rather than guessing one by hand:

$ cibuildmp --build "mpy6.3-* v1.29.0-manylinux* v1.29.0-esp32-*" \
    --print-build-identifiers

build = "mpy6.3-*"                        # every arch, one natmod ABI, newest verified tag
build = "mpy6.2-* mpy6.3-*"                # two ABIs in one invocation
build = "v1.29.0-manylinux*"               # every native unix cell, one tag
skip  = "*_ppc64le *_s390x *_riscv64"      # drop the emulated-everywhere cells
build = "v1.29.0-esp32-ESP32_GENERIC"      # exactly one board

[override."*-armv7emsp"]
extra-make-args = ["MP_BCLIBC_PRECISION=single"]

{...,...} brace expansion works inside a pattern (build = "*-{x64,armv6m}"), matching shell glob syntax. A build/skip pattern that can never match any real identifier is a load-time error, not a silent no-op.

When a build glob names no specific MicroPython tag, natmod narrows the match to the newest tag this project has verified as stable for that ABI — name one explicitly (mpy6.3-v1.29.0-*) to pin it yourself. Usermod has no equivalent narrowing: every real (port, tag, arch/board) row already carries its own explicit tag.

--keep-going and the JSON build report

The default is fail-fast: the first target to fail stops the whole invocation, and nothing selected after it is even attempted — the same behaviour cibuildwheel itself has, unconditionally, with no keep-going concept of its own. --keep-going (record 0063) is a deliberate cibuildmp divergence for the opposite case — a --build glob wide enough to span a real coverage sweep, where the point is to find out every target's own outcome rather than stop at the first one that fails.

Every attempted target — success or failure, --keep-going or not — is written to a JSON report, one file per invocation, under ~/.cache/cibuildmp/reports/ by default (CIBMP_REPORT_PATH to redirect it). Each entry carries the identifier, how long it took, and either the built artifact's directory/size/file listing or the error that stopped it:

{
  "generated_at": "2026-08-29T12:00:00+00:00",
  "total_duration": 12.4,
  "built": 1,
  "failed": 1,
  "results": [
    {
      "identifier": "v1.29.0-manylinux_2_28_x86_64",
      "duration": 9.1,
      "error": null,
      "output_dir": "mpyhouse/v1.29.0-manylinux_2_28_x86_64",
      "size": 1048576,
      "files": ["micropython-v1.29.0-manylinux_2_28_x86_64"]
    },
    {
      "identifier": "v1.29.0-qemu-POWERNV9",
      "duration": 3.3,
      "error": "make: *** [firmware.elf] Error 1",
      "output_dir": null,
      "size": null,
      "files": []
    }
  ]
}

Target support

Natmod, per arch

All ten ARCH= values py/dynruntime.mk accepts, all baked into one docker/natmod.Dockerfile image (linux/amd64, pulled from ghcr.io/ballistics-lab/natmod) — natmod builds no bare-host toolchain of any kind any more, x86's 32-bit multilib included, which is exactly what makes it buildable on an arm64 runner too. Adopted in all three consuming repos and verified on real CI, arch by arch, not just --dry-run.

Usermod, per port/arch

Upstream MicroPython has 20 ports (ports/* in a real checkout); every one is listed below for orientation, not just the ones this project covers. resources/build-platforms.toml carries independently-verified (tag, arch/board) rows for 15 of them; only 6 (unix, windows, qemu, webassembly, esp32, rp2) have a real build driver wired into the CLI at all — the other 9 have verified facts a config can already name, but nothing yet to actually build them. Every ✅ row below is live-verified against a real MicroPython checkout, including a real custom USER_C_MODULES module, Docker-only. unix, windows, webassembly and qemu are exercised through build-examples.yml's own small integration smoke test on every push, producing genuine linked binaries with their executable bit intact (unix: one native image per arch/libc; windows/webassembly/qemu: one image each — qemu's own v1.29.0-qemu-MPS2_AN385 runs in its own matrix leg rather than sharing a job with already-proven cells, since it was the first build ever run through that path). esp32 and rp2 are not in that smoke test, but not because either is unproven -- both are exercised far more broadly, on every pull request, through test-all-platforms.yml's own real matrix (bin/plan_test_matrix.py, record 0065): 83 real esp32 identifiers and 74 real rp2 ones, every board/tag row resources/build-platforms.toml carries, not a spot check. Both build through the exact same cibuildmp CLI/action every other ✅ row does -- esp32's own composite action (tracker item [0038]) was only ever needed while build_esp32() still provisioned onto the bare host; record 0028 moved it fully into esp_idf_base (Docker) on 2026-08-28, and nothing in this project still depends on that composite action's own toolchain-install path. rp2's own driver landed the next day (record 0060), live-verified against examples/template first and now carrying its own share of every test-all-platforms.yml run since.

No Windows or macOS host is needed for any of the ✅/⚠️ usermod targets above, windows's own three arches included — every toolchain there is either already on a Linux host or downloads/apt-installs onto one.

Conventions this repo assumes

A module following the natmod/usermod layout looks like:

natmod/
  Makefile              # includes py/dynruntime.mk, dispatches on ARCH=
usermod/
  micropython.cmake
  micropython.mk
  manifest.py

build-natmod only assumes natmod/Makefile (or whatever natmod_dir points at) accepts ARCH= and MPY_DIR= and has a dist target that drops the built .mpy under build/<arch>*/. Nothing here assumes a specific module name, precision scheme, or test framework — those stay in the consuming repo.

One more requirement for the cibuildmp CLI specifically: scope dynruntime.mk's BUILD variable by $(ARCH) — BUILD = .obj/$(ARCH) before the include, kept outside build/ so it does not collide with the dist output the CLI globs for (see examples/template/natmod/Makefile). cibuildmp runs every selected target sequentially in one natmod/ tree, and dynruntime.mk defaults BUILD ?= build unscoped, so without this a second ARCH= in the same invocation finds the previous arch's own object files "up to date" and skips rebuilding — the merged .mpy silently stays the first arch's binary. cibuildmp catches this itself (a header-arch verification step fails loudly instead), but scoping BUILD avoids paying for the failed build at all.

If the module also builds rv32imc with more than one arch-flags value in the same invocation, BUILD needs $(ARCH_FLAGS) folded in too — BUILD = .obj/$(ARCH)$(if $(ARCH_FLAGS),+$(ARCH_FLAGS)), for the same reason on that second axis.

None of this cares what produced the .c files SRC lists — examples/wasm2mpy compiles WebAssembly to C via wasm2c in a Makefile rule before the same dynruntime.mk flow takes over.

Roadmap

docs/0000-TRACKER.md is the plan of record: the decisions taken and why (in docs/records/), what's implemented, what's deliberately deferred. docs/BACKLOG.md is now just a short redirect into that scheme, not itself the plan.

Natmod is done end to end — target selection, MicroPython/mpy-cross provisioning, and the build itself, all ten arches now running inside one pulled docker/natmod.Dockerfile image rather than a host-side toolchain resolver (that resolver, and its own --toolchain flag, are deleted) — verified on real CI in all three consuming repos, not just --dry-run. Usermod's own build drivers are wired into the CLI too (see Target support above): unix/windows/webassembly run live through the real action.yml, all three in one invocation. What's still open is the third consuming-repo step: none of micropython-bclibc/a7p/micropython-wasm3 has repinned its own usermod workflow to the cibuildmp CLI yet. Until one does, the composite actions stay the supported path for it.

Composite actions

The pre-CLI building blocks — one GitHub Action per build step (fetch-micropython, build-natmod, build-usermod-unix/-windows/ -webassembly/-rp2040/-armv7m/-esp32, …). Still fully supported for CI, but no longer where new work starts — new usermod ports and arches land in the CLI's own usermod/build_<port>.py first. Full input/output reference and a usage example: docs/ACTIONS.md.

Versioning

Pin consumers to a tag, not @main and not a commit SHA — bumping the tag a consumer references is a deliberate, visible edit in that repo, same as bumping any other CI dependency.

The cibuildmp package and the actions share one version. v0.3.0 is the first tag where the CLI actually builds a module, not just plans it; it continues micropython-native-ci's version line rather than restarting it, since this repo absorbed that one (its consumers have since repinned; the old repo is now archived). See CHANGELOG.md for the full history.

The root action.yml installs cibuildmp from its own checkout rather than from PyPI (uv tool install "$GITHUB_ACTION_PATH", already the pinned ref's own source, checked out by GitHub Actions before any step runs), so the tool that runs on CI is always exactly the ref you pinned, with no index to keep in sync. Running it yourself with uv tool install cibuildmp/pip install cibuildmp instead pulls whatever's newest on PyPI unless you pin a version there too.