buildgen - generate your build-system

A build system generator package supporting Makefile, CMake, and scikit-build-core project definitions.

Note: the buildgen test suite generates, build and tests every one of its recipes.

Installation

pip install buildgen

Quick Start

# Create a C++ project
buildgen new myapp

# Create a Python extension with pybind11
buildgen new myext -r py/pybind11

# List available recipes
buildgen list

Features

• Makefile Generation: Programmatic Makefile creation with variables, pattern rules, conditionals
• CMake Generation: Full CMakeLists.txt generation with find_package, FetchContent, install rules
• Cross-Generator: Define project recipes once in JSON/YAML, generate build systems
• CMake Frontend: Use CMake as build system with convenient Makefile frontend
• Project Templates: Quick-start templates for common project types
• scikit-build-core Templates: Python extension project scaffolding (pybind11, cython, nanobind, C)
• Template Customization: Override templates per-project, per-user, or via environment variable (Mako syntax)
• Configurable Project Recipes: 2-step JSON/YAML recipes which include options and which are rendered to generate the project infrastructure.
• User Configuration: Global ~/.buildgen/config.toml for author identity and project defaults (license, language standards, Python version, env tool)
• Dependency Version Resolution: Automatically resolves latest dependency versions from PyPI at generation time, with offline fallback and per-user version pinning
• Reproducible Generation: buildgen.lock records resolved dependency versions; --offline generates without network access
• Safe Generation: --dry-run previews output paths, and existing files are never replaced without --force
• Toolchain Profiles and Presets: Named compiler, cross-compilation, and toolchain-file settings, with opt-in Debug/Release CMakePresets.json
• Configuration Validation: buildgen validate reports unknown keys, duplicate targets, and dependency errors before anything is written
• Toolchain Diagnostics: buildgen doctor reports detected tools and versions, with --json for CI

Usage

CLI

# Create projects from recipes
buildgen new myapp -r cpp/executable
buildgen new mylib -r c/static
buildgen new myext -r py/pybind11

# List available recipes
buildgen list

# Generate build files from a config file
buildgen generate --from project.json

# Preview output paths without writing, then replace existing files
buildgen generate --from project.json --dry-run
buildgen generate --from project.json --force

# Generate CMakePresets.json alongside CMakeLists.txt
buildgen generate --from project.json --presets --profile clang

# Check the config before generating anything
buildgen validate project.json

# Report detected tools, or the ones a recipe needs
buildgen doctor
buildgen doctor --json
buildgen doctor --recipe py/pybind11

# Record resolved dependency versions for reproducible generation
buildgen lock -o buildgen.lock

# Test recipe generation and building
buildgen test --all

# Direct Makefile generation (advanced)
buildgen makefile generate -o Makefile --targets "all:main.o:"

# Direct CMake generation (advanced)
buildgen cmake generate -o CMakeLists.txt --project myapp --cxx-standard 17

Python API

from buildgen import ProjectConfig, TargetConfig, DependencyConfig

# Load from config file
config = ProjectConfig.load("project.json")
config.generate_all()  # Creates Makefile and CMakeLists.txt

# Or build programmatically
config = ProjectConfig(
    name="myproject",
    version="1.0.0",
    cxx_standard=17,
    compile_options=["-Wall", "-Wextra"],
    dependencies=[
        DependencyConfig(name="Threads"),
        DependencyConfig(
            name="fmt",
            git_repository="https://github.com/fmtlib/fmt.git",
            git_tag="10.1.1",
        ),
    ],
    targets=[
        TargetConfig(
            name="mylib",
            target_type="static",
            sources=["src/lib.cpp"],
            include_dirs=["include"],
        ),
        TargetConfig(
            name="myapp",
            target_type="executable",
            sources=["src/main.cpp"],
            link_libraries=["mylib", "fmt::fmt"],
            install=True,
        ),
    ],
)
config.generate_all()

The two generators are driven by the same fields, but the Makefile back end is the one that has to make language decisions explicitly:

• languages, the source file extensions, and c_standard / cxx_standard decide which pattern rules and -std= flags are emitted. A target whose sources are all .c links with $(CC); anything containing C++ links with $(CXX).
• Declaring any shared target adds -fPIC to every compile, because objects are built by a single global pattern rule.
• Library names are normalized on the way to the link line: a CMake-style fmt::fmt becomes -lfmt, and a value that is already a flag (-ldl) is passed through untouched.
• include_dirs and link_dirs are validated against the filesystem, so the directories must exist before generate --from runs.

Two behaviors worth knowing when driving MakefileGenerator directly:

• Targets are written in the order they are added, so the first target you add becomes make's default goal. Add your all aggregate first.
• Include and link directories are validated when added. A path that does not exist raises ValueError; a $(VAR) reference is resolved when the variable is already known and otherwise accepted and left for make.

CMake with Makefile Frontend

Generate CMake as the build system with a Makefile that wraps cmake commands:

config.generate_cmake_with_frontend(
    build_dir="build",
    build_type="Release",
)

This creates:

• CMakeLists.txt - The actual build logic
• Makefile - Convenience wrapper with targets:

make              # Configure and build
make build        # Same as above
make configure    # Run cmake configure only
make clean        # Remove build directory
make rebuild      # Clean and rebuild
make install      # Install the project
make test         # Run tests with ctest
make myapp        # Build specific target
make help         # Show available targets

# Override defaults
make BUILD_TYPE=Debug
make BUILD_DIR=cmake-build
make CMAKE_FLAGS="-DFOO=bar"

Project Configuration

JSON Format

{
    "name": "myproject",
    "version": "1.0.0",
    "cxx_standard": 17,
    "compile_options": ["-Wall", "-Wextra"],
    "dependencies": [
        "Threads",
        {"name": "OpenSSL", "required": true},
        {
            "name": "fmt",
            "git_repository": "https://github.com/fmtlib/fmt.git",
            "git_tag": "10.1.1"
        }
    ],
    "targets": [
        {
            "name": "mylib",
            "type": "static",
            "sources": ["src/lib.cpp"],
            "include_dirs": ["include"],
            "install": true
        },
        {
            "name": "myapp",
            "type": "executable",
            "sources": ["src/main.cpp"],
            "link_libraries": ["mylib", "Threads::Threads"],
            "install": true
        }
    ]
}

YAML Format

name: myproject
version: 1.0.0
cxx_standard: 17

compile_options:
  - -Wall
  - -Wextra

dependencies:
  - Threads
  - name: OpenSSL
    required: true
  - name: fmt
    git_repository: https://github.com/fmtlib/fmt.git
    git_tag: 10.1.1

targets:
  - name: mylib
    type: static
    sources:
      - src/lib.cpp
    include_dirs:
      - include
    install: true

  - name: myapp
    type: executable
    sources:
      - src/main.cpp
    link_libraries:
      - mylib
      - Threads::Threads
    install: true

Dependency Providers

A dependency's provider states how it is obtained. Without it, the provider is inferred: a dependency carrying git_repository or url is built from source, anything else is looked up on the system.

Provider	CMake output	Makefile output
system (default)	find_package(...)	-l<name>
cmake	find_package(...)	-l<name>
fetchcontent	FetchContent_Declare(...)	no link flag

{
    "dependencies": [
        {"name": "OpenSSL", "provider": "system", "components": ["SSL"]},
        {
            "name": "fmt",
            "provider": "fetchcontent",
            "git_repository": "https://github.com/fmtlib/fmt.git",
            "git_tag": "10.1.1"
        }
    ]
}

fetchcontent requires git_repository or url; system and cmake reject both. buildgen validate reports either mistake.

Toolchain Profiles and CMake Presets

A profile is a named set of compiler, flag, and cross-compilation settings. Profiles keep build configuration separate from project shape, so one project definition can target several environments.

{
    "name": "myproject",
    "profile": "clang",
    "profiles": {
        "clang": {
            "cc": "clang",
            "cxx": "clang++",
            "compile_options": ["-Wall", "-Wextra"]
        },
        "arm64": {
            "toolchain_file": "cmake/arm64.cmake",
            "cmake_variables": {"CMAKE_SYSTEM_NAME": "Generic"}
        }
    }
}

The profile key selects the active profile; buildgen generate --profile <name> overrides it for one run. The active profile's compilers and flags are merged into the generated Makefile and CMakeLists.txt. Compiler, toolchain-file, and cmake_variables entries are written above project(), because CMake reads them while project() runs and ignores them afterwards.

--presets writes a CMakePresets.json with Debug and Release configure, build, and test presets. Every preset sets CMAKE_EXPORT_COMPILE_COMMANDS=ON for language servers and IDEs, and carries the active profile's cache variables.

buildgen generate --from project.json --presets --profile clang
cmake --preset debug && cmake --build --preset debug

Validation and Safe Overwrite

buildgen validate project.json checks a configuration without writing anything. It reports unknown top-level keys, missing or duplicate target names, invalid target types, provider mismatches, and unknown profile names. It exits nonzero when any check fails, so it works as a CI gate. buildgen generate runs the same checks before generating.

Generation never replaces an existing file silently:

• --dry-run lists the paths that would be written, marking each create or update, and writes nothing.
• Without --force, an existing output file aborts the run.
• --force replaces existing files.

These flags apply to buildgen new, buildgen render, and buildgen generate.

Project Recipes

Recipes use a category/variant naming convention:

buildgen list

C++ Recipes (CMake + Makefile frontend):

Recipe	Description
cpp/executable	Single executable
cpp/static	Static library
cpp/shared	Shared library (-fPIC)
cpp/header-only	Header-only library
cpp/library-with-tests	Library + tests
cpp/app-with-lib	App with internal library
cpp/full	Library + app + tests

C Recipes (CMake + Makefile frontend):

Recipe	Description
c/executable	Single executable
c/static	Static library
c/shared	Shared library (-fPIC)
c/header-only	Header-only library
c/library-with-tests	Library + tests
c/app-with-lib	App with internal library
c/full	Library + app + tests

Python Extension Recipes (scikit-build-core):

Recipe	Description
py/pybind11	C++ extension using pybind11
py/pybind11-flex	Pybind11 extension with optional Catch2/GTest tests + CLI
py/nanobind	C++ extension using nanobind
py/cython	Extension using Cython
py/cext	C extension (Python.h)

Pure Python Recipes (uv_build, no native build):

Recipe	Description
py/nodeps	Pure-Python package with no runtime dependencies

Python Extension Projects

Generate complete Python extension projects with scikit-build-core:

# Create a pybind11 extension project
buildgen new myext -r py/pybind11

# Use traditional virtualenv instead of uv
buildgen new myext -r py/pybind11 --env venv

This creates a complete project structure:

myext/
  pyproject.toml      # scikit-build-core configuration
  CMakeLists.txt      # CMake build instructions
  Makefile            # Convenience wrapper
  src/myext/
    __init__.py       # Python package
    _core.cpp         # C++ extension source
  tests/
    test_myext.py     # pytest tests

The generated Makefile provides convenient commands (using uv by default):

make sync     # Initial setup (uv sync)
make build    # Rebuild extension after code changes
make test     # Run tests (uv run pytest)
make wheel    # Build wheel distribution
make clean    # Remove build artifacts

For traditional virtualenv workflows, use --env venv to generate pip/python commands instead.

Pure Python Projects

The py/nodeps recipe generates a pure-Python package with no runtime dependencies and no compiler in the loop:

buildgen new mypkg -r py/nodeps

mypkg/
  pyproject.toml      # uv_build backend, dependencies = []
  Makefile            # Convenience wrapper
  src/mypkg/
    __init__.py       # Python package
    core.py           # Example module (standard library only)
    py.typed          # PEP 561 marker
  tests/
    test_mypkg.py     # pytest tests
  .github/workflows/  # ci.yml + publish.yml

Differences from the extension recipes: uv_build replaces scikit-build-core, there is no CMakeLists.txt, the CI build leg runs on a single runner (one py3-none-any wheel serves every platform), and publishing uses a plain sdist/wheel workflow rather than cibuildwheel. Development tooling (pytest, ruff, mypy) lives in the dev dependency group, so it is not part of the published distribution. The generated test suite asserts that the installed distribution declares no runtime requirements, so the invariant is enforced rather than merely documented.

The py/pybind11-flex recipe additionally drops a project.flex.json whose options block toggles the native Catch2/GTest harness and the optional embedded CLI. Those three values are the only inputs to the render: they are baked into pyproject.toml's [tool.scikit-build.cmake.define] table and into the option() defaults in CMakeLists.txt. To explore a different combination, edit the block and run buildgen render again -- re-running cmake by itself will not pick up a change, because scikit-build-core re-applies its own defines on every build. The rendered project.json records the equivalent cmake -D flags for reference.

Configurable Recipe Workflow

For recipes marked as configurable (like py/pybind11-flex), project creation is a two-step flow:

buildgen new myflex -r py/pybind11-flex      # emits myflex/project.flex.json
# edit myflex/project.flex.json (env, test framework, CLI toggle)
buildgen render myflex/project.flex.json     # renders full project based on options

buildgen render produces a standard config (project.json or .yaml, depending on the source filename) inside the generated project with all placeholders resolved, while the original project.flex.json stays wherever you edited it for future re-runs. You can run buildgen render from within the project directory as long as you point to the flex file. Use --env venv on buildgen render to override the config’s environment choice without editing the JSON/YAML.

User Configuration

Set your identity and project defaults globally via ~/.buildgen/config.toml:

# Create the config file with a commented template
buildgen config init

# View current config
buildgen config show

# Print config file path
buildgen config path

Config Format

[user]
name = "Your Name"
email = "you@example.com"

[defaults]
license = "MIT"
cxx_standard = 17
c_standard = 11
python_version = "3.10"
env_tool = "uv"

[deps]
# Pin dependency versions used in generated projects.
# These override both PyPI resolution and bundled defaults.
# Omitted packages are resolved normally.
# ruff = "0.14.0"
# mypy = "1.18.0"

What the config affects

• user.name / user.email -- Populates the LICENSE copyright holder and [[project.authors]] in generated pyproject.toml files.
• defaults.license -- Sets the license identifier in pyproject.toml (default: MIT).
• defaults.cxx_standard -- Sets CMAKE_CXX_STANDARD in C++ CMakeLists.txt templates (default: 17).
• defaults.c_standard -- Sets CMAKE_C_STANDARD in C CMakeLists.txt templates (default: 11).
• defaults.python_version -- Sets requires-python in pyproject.toml (default: 3.10).
• defaults.env_tool -- Fallback environment tool (uv or venv) when --env is not explicitly passed on the command line.
• deps.<package> -- Pin a specific version for a dev dependency or build-system requirement. Overrides both PyPI resolution and bundled defaults.

All defaults are optional. Without a config file, templates use their built-in fallback values.

Dependency Version Resolution

Generated Python projects (py/* recipes) include dev dependencies (ruff, mypy, pytest, etc.) and build-system requirements (scikit-build-core). By default, buildgen resolves the latest versions of these packages from PyPI at generation time, so scaffolded projects start with current tooling.

The resolution order is:

1. User config [deps] -- Pinned versions in ~/.buildgen/config.toml always win
2. PyPI latest -- Queried at generation time (default behavior)
3. Bundled defaults -- Fallback when offline or on error

To skip PyPI resolution entirely (e.g., in CI or offline environments):

buildgen new myext -r py/pybind11 --no-update-deps

This uses the bundled default versions (still overridden by any [deps] pins in user config).

Lock Files

buildgen lock writes the resolved versions to a buildgen.lock file, so the same versions are used on another machine or at a later date.

buildgen lock -o myext/buildgen.lock     # resolve from PyPI
buildgen lock --offline -o buildgen.lock # use bundled defaults, no network

A buildgen.lock in the output directory takes precedence over a PyPI query, with or without --offline. Packages the lock does not name fall back to the bundled defaults, so an older lock still generates a complete project. The full resolution order is:

1. User config [deps] -- pins in ~/.buildgen/config.toml
2. buildgen.lock in the output directory
3. PyPI latest, unless --offline or --no-update-deps is set
4. Bundled defaults

The file records lock_version, the buildgen version that wrote it, the recipe, and the sorted dependency versions. buildgen rejects a lock whose lock_version it does not know rather than generating from a format it cannot read.

buildgen doctor reports the tools generated projects need (cmake, make, compilers, python, uv). With --recipe, it exits nonzero when a tool that recipe requires is missing; --json emits the same result for CI scripts.

Template Customization

Templates can be customized without modifying buildgen. Override files are resolved in this order (first match wins):

1. $BUILDGEN_TEMPLATES/{recipe}/ - Environment variable (for CI/CD)
2. .buildgen/templates/{recipe}/ - Project-local overrides
3. ~/.buildgen/templates/{recipe}/ - User-global defaults
4. Built-in templates

Template Commands

# List available templates and show which have overrides
buildgen templates list

# Copy templates for local customization
buildgen templates copy py/pybind11

# Copy to global location for user-wide defaults
buildgen templates copy py/pybind11 --global

# Show where each template file is resolved from
buildgen templates show py/pybind11

Customizing Templates

1. Copy the templates you want to customize:

   buildgen templates copy py/pybind11
   

2. Edit the .mako files in .buildgen/templates/py/pybind11/:

   # Customize pyproject.toml template
   edit .buildgen/templates/py/pybind11/pyproject.toml.mako
   

3. Generate projects - your customizations will be used:

   buildgen new myext -r py/pybind11
   

Templates use Mako syntax with ${variable} for substitution.

Per-File Overrides

You can override individual files while keeping others from built-in templates. For example, to customize only pyproject.toml:

mkdir -p .buildgen/templates/py/pybind11
cp $(buildgen templates show py/pybind11 | grep pyproject) .buildgen/templates/py/pybind11/
# Edit your local copy

Limitation: Mako includes are not resolved through the override chain

Override resolution applies to the files listed by buildgen templates show -- the entries in a recipe's file map. It does not apply to templates pulled in by a Mako <%include> directive from inside another template. The py/* recipes use one such include: every pyproject.toml.mako includes common/pyproject.base.toml.mako, which Mako resolves relative to the including template's own directory and the built-in py/ root only.

Practical consequence: dropping a file at ~/.buildgen/templates/py/common/pyproject.base.toml.mako has no effect and produces no warning. To customize the shared pyproject base, override the recipe's pyproject.toml.mako instead -- either replace the include with your own content, or keep the include and place your modified base alongside your override at .buildgen/templates/py/pybind11/common/pyproject.base.toml.mako, which Mako will find because it sits under the overriding template's directory.

Low-Level API

For fine-grained control, use the generators directly:

from buildgen import MakefileGenerator, CMakeListsGenerator

# Makefile
gen = MakefileGenerator("Makefile")
gen.add_cxxflags("-Wall", "-std=c++17")
gen.add_target("all", deps=["myapp"])  # declared first, so it is the default goal
gen.add_target("myapp", "$(CXX) $(CXXFLAGS) -o $@ $^", deps=["main.o"])
gen.add_pattern_rule("%.o", "%.cpp", "$(CXX) $(CXXFLAGS) -c $< -o $@")
gen.add_phony("all", "clean")
gen.generate()

# CMake
gen = CMakeListsGenerator("CMakeLists.txt")
gen.set_project("myapp", version="1.0.0")
gen.set_cxx_standard(17)
gen.add_find_package("Threads", required=True)
gen.add_executable("myapp", ["src/main.cpp"], link_libraries=["Threads::Threads"])
gen.generate()

Development

make test        # Run tests
make lint        # Run ruff check
make coverage    # Coverage report

Credits

• Template rendering powered by an embedded version of Mako Templates (MIT License)
• Originally inspired by prior work in shedskin.makefile in the shedskin project

License

GPL-3.0-or-later