Nuwa Build

Compile Nim into Python extensions. Docs: Nuwa documentation. Changelog: CHANGELOG.md.

First hour

1. Install a Nim compiler and confirm nim --version works.
2. pip install nuwa-build (CPython 3.10 or newer).
3. nuwa new hello && cd hello
4. nuwa develop
5. python example.py

nuwa new installs nimpy and nuwa_sdk from nimble-deps. You do not need a separate nimble install nimpy for a new project.

Mark every Nim procedure that should appear in Python with {.nuwa_export.}. Without that pragma the project still compiles, and the function is invisible from Python.

Optional extras:

pip install "nuwa-build[watch]"     # nuwa watch
pip install "nuwa-build[notebook]"  # Jupyter/IPython magics

Support matrix

Features

• Zero Configuration: Works out of the box with sensible defaults
• Multi-file Projects: Compile multiple Nim files into a single Python extension
• Flexible Configuration: Configure via pyproject.toml or CLI arguments
• PEP 517/660 Compatible: Build wheels and source distributions
• Build Command: nuwa build for easy local wheel creation
• Cross-Platform CI: Template includes GitHub Actions workflow with manylinux support
• Editable Installs: pip install -e . support for development
• Watch Mode: Auto-recompile on file changes with nuwa watch
• Auto Dependencies: Automatically install Nimble packages before build
• Testing Support: Includes pytest tests in project template
• Validation: Validates configuration and provides helpful error messages
• Proper Platform Tags: Generates correct wheel tags for your platform

Installation

pip install nuwa-build

See First hour for Nim, extras, and {.nuwa_export.}.

Quick Start

Create a New Project

nuwa new my_project
cd my_project

This creates:

my_project/
├── pyproject.toml           # Python project config
├── nim/                     # Nim source files
│   ├── my_project_lib.nim  # Main entry point (filename = module name)
│   └── helpers.nim          # Additional modules
├── my_project/              # Python package
│   ├── __init__.py          # Package wrapper
│   └── my_project_lib.so    # Compiled extension (generated)
├── tests/                   # Test files
│   └── test_my_project.py   # Pytest tests
├── example.py               # Example/test file
└── README.md

Build and Test

# Compile debug build
nuwa develop

# Compile release build
nuwa develop --release

# Run example
python example.py

# Run tests (requires pytest)
pip install pytest
pytest

Note: No pip install -e . needed due to flat project layout. You can run python example.py and pytest directly after compiling the default template project.

🤖 AI-Assisted Development

This project includes a Nuwa Build agent skill that teaches compatible AI coding agents how to work with Nuwa Build.

Why use it?

Standard LLMs often assume Python extensions require setup.py or pip install -e .. The skill provides the correct context to:

• Understand the flat layout: Knows that .so files are generated directly in the package directory.
• Use correct commands: Enforces nuwa develop and nuwa watch instead of standard pip commands.
• Write correct Nim: Reminds the agent to use include for shared libraries and {.nuwa_export.} for bindings (which automatically generates type stubs).

How to use:

Create a nuwa-build directory in the skill location supported by your coding agent and copy skills/SKILL.md into it. If the agent does not support skills, provide that file as project instructions or conversation context.

Watch Mode

For development, use watch mode to automatically recompile when you change Nim files:

pip install "nuwa-build[watch]"

# Watch for changes and auto-recompile
nuwa watch

# Watch with tests after each compile
nuwa watch --run-tests

# Watch in release mode
nuwa watch --release

Jupyter Notebook Support

Nuwa-Build includes a Jupyter magic command for compiling Nim code directly in notebooks.

Install the notebook integration with pip install "nuwa-build[notebook]".

Usage

# Load the extension
%load_ext nuwa_build.magic

# Compile Nim code in a cell
%%nuwa
proc add(a, b: int): int {.nuwa_export.} =
    return a + b

# Use the function immediately (no import needed!)
add(1, 2)  # Output: 3

Caching

Nuwa automatically caches compiled modules in .nuwacache/:

• Cache hits: Re-running cells uses cached compilation (fast!)
• Cache misses: New code or different flags trigger recompilation
• Persistent: Cache survives kernel restarts

Cache Management

# Show cache statistics
%nuwa_cache_info
# Output: 📊 Cache: 3 modules, 123.4 KB
#         📍 Location: /path/to/.nuwacache

# Clear cache (e.g., to force rebuild or free space)
%nuwa_clean
# Output: 🧹 Cleared cache: .nuwacache

Or manually delete the .nuwacache/ folder.

.gitignore

Add to your .gitignore:

.nuwacache/

Compiler Flags

Pass Nim compiler flags on the magic line:

%%nuwa -d:release --opt:speed
proc optimized_func(n: int): int {.nuwa_export.} =
    # Optimized implementation
    ...

Different flags create different cache entries (code hash includes flags).

Example

# Cell 1: Load extension
%load_ext nuwa_build.magic

# Cell 2: Compile simple function
%%nuwa
proc greet(name: string): string {.nuwa_export.} =
    return "Hello, " & name & "!"

# Cell 3: Use immediately
print(greet("World"))  # Output: Hello, World!

# Cell 4: Multiple functions
%%nuwa
proc add(a, b: int): int {.nuwa_export.} = a + b
proc multiply(a, b: int): int {.nuwa_export.} = a * b

print(add(5, 3))      # 8
print(multiply(4, 7)) # 28

# Cell 5: With compiler flags
%%nuwa -d:release
proc fibonacci(n: int): int {.nuwa_export.} =
    if n <= 1:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

print(fibonacci(10))  # 55 (optimized)

Install and Distribute

# Build a wheel for distribution
nuwa build

# Build and install locally
pip install dist/*.whl

# Build using standard Python tools (alternative)
pip install . --no-build-isolation

# Build source distribution
python -m build --sdist

Continuous Integration with GitHub Actions

The nuwa new template includes a pre-configured GitHub Actions workflow (.github/workflows/publish.yml) for automated cross-platform wheel building and PyPI publishing.

What's Included

The workflow uses a custom composite action (martineastwood/nuwa-build-action@v1) that integrates with cibuildwheel to build wheels across:

• Platforms: Linux (manylinux), macOS, Windows
• Python versions: 3.10, 3.11, 3.12, 3.13, 3.14
• Architectures: The native architecture of each selected GitHub-hosted runner

Each wheel is installed into cibuildwheel's test environment and checked with an import of the configured Python module. The generated matrix covers regular CPython builds; free-threaded builds are intentionally excluded until they are verified with Nimpy.

How It Works

The custom action handles platform-specific Nim compiler installation:

First-Time Setup

1. Configure Trusted Publishing on PyPI:

   • Go to https://pypi.org/manage/account/publishing/
   • Add a new publisher with:
     • PyPI Project Name: Your package name
     • Owner: Your GitHub username/organization
     • Repository name: Your repository name
     • Workflow name: publish.yml

2. Push a version tag to trigger the workflow:

   git tag v1.0.0
   git push origin v1.0.0
   

Manual Workflow Trigger

You can also manually trigger the workflow from the GitHub Actions tab in your repository, useful for testing CI before release.

Customization

To customize the build (e.g., different Nim version or cibuildwheel version), edit .github/workflows/publish.yml:

- name: Build wheels
  uses: martineastwood/nuwa-build-action@v1
  with:
    nim-version: "2.2.10"     # Nim version to install
    cibw-version: "4.2.0"     # cibuildwheel version

Project Structure

Nuwa uses a simple flat layout for easy development:

project/
├── pyproject.toml              # Configuration
├── nim/                        # Nim source files
│   ├── my_package_lib.nim     # Main entry point (determines module name)
│   └── helpers.nim             # Additional modules
├── my_package/                 # Python package
│   ├── __init__.py             # Package wrapper (can add Python code)
│   └── my_package_lib.so       # Compiled Nim extension (generated)
└── tests/
    └── test_my_package.py      # Pytest tests

The compiled extension is named {module_name}_lib.so to avoid conflicts with the Python package. Your __init__.py imports from it and can add Python wrappers.

Configuration

pyproject.toml

Configure your project in the [tool.nuwa] section:

[build-system]
requires = ["nuwa-build"]
build-backend = "nuwa_build"

[project]
name = "my-package"
version = "0.1.0"

[tool.nuwa]
# Nim source directory (default: "nim")
nim-source = "nim"

# Python module name (default: derived from project name)
module-name = "my_package"

# Internal library name (default: "{module_name}_lib")
lib-name = "my_package_lib"

# Entry point file (default: "{lib_name}.nim")
entry-point = "my_package_lib.nim"

# Output location: "auto", "src", or explicit path
output-location = "auto"

# Additional Nim compiler flags (optional)
nim-flags = []

# Nimble dependencies (auto-installed before build)
nimble-deps = ["nimpy", "cligen@>=1.0.0"]

# Windows packaging safeguards (optional)
# Statically link MinGW runtimes on Windows (default: true)
windows-static-linking = true

# Bundle adjacent DLLs alongside the compiled extension (default: true)
bundle-adjacent-dlls = true

# Allow MANIFEST.in to include compiled binaries like .pyd/.dll/.so (default: false)
allow-manifest-binaries = false

# Build profiles (predefined compiler flag sets)
[tool.nuwa.profiles.dev]
nim-flags = ["-d:debug", "--debugger:native", "--linenos:on"]

[tool.nuwa.profiles.release]
nim-flags = ["-d:release", "--opt:speed", "--stacktrace:off"]

[tool.nuwa.profiles.bench]
nim-flags = ["-d:release", "--opt:speed", "--stacktrace:on"]

[tool.nuwa.profiles.size]
nim-flags = ["-d:release", "--opt:size"]

Configuration Options

Note: lib-name determines the compiled extension filename. With Nimpy, keep the entry-point basename aligned with lib-name (the generated default does this) so the extension's Python initialization symbol matches its filename.

Windows note: By default, MinGW runtime libraries are linked statically on Windows, so wheels don’t depend on libgcc_s_seh-1.dll, libstdc++-6.dll, or libwinpthread-1.dll. Set windows-static-linking = false only if you intend to manage these dependencies outside the wheel.

MANIFEST note: By default, compiled binaries like .pyd, .dll, .so, and .dylib are excluded even if listed in MANIFEST.in. Set allow-manifest-binaries = true to include them explicitly.

Build Profiles

Build profiles allow you to define preset compiler flag configurations for different scenarios:

[tool.nuwa.profiles.dev]
nim-flags = ["-d:debug", "--debugger:native", "--linenos:on"]

[tool.nuwa.profiles.release]
nim-flags = ["-d:release", "--opt:speed", "--stacktrace:off"]

[tool.nuwa.profiles.bench]
nim-flags = ["-d:release", "--opt:speed", "--stacktrace:on"]

Usage:

nuwa develop --profile dev
nuwa build --profile release
nuwa watch --profile bench

Flag precedence:

1. Base nim-flags from [tool.nuwa]
2. Profile flags (appended)
3. CLI --nim-flag arguments (applied last, for maximum flexibility)

Shell Completion

Nuwa supports shell completion for bash, zsh, and tcsh via the shtab library.

Installation:

# Install shtab (optional dev dependency)
pip install shtab

# Generate and install completions
nuwa --print-completion bash > ~/.local/share/bash-completion/completions/nuwa
nuwa --print-completion zsh > ~/.zfunc/_nuwa
nuwa --print-completion tcsh > ~/.nuwa-completion.tcsh

For zsh users, add to your ~/.zshrc:

# Add completion function path
fpath=(~/.zfunc $fpath)
autoload -U compinit && compinit

Completions work automatically for all commands, flags, and will suggest files/directories where appropriate.

Package Data

Nuwa automatically includes all non-Python files in your package directory (config files, data files, templates, etc.) when building wheels.

Automatic Inclusion (Default)

By default, all files in your package directory are included in the wheel, except for:

• Python cache (__pycache__, *.pyc, *.pyo)
• Compiled extensions (.so, .pyd, .dll - added separately)
• Version control (.git, .hg, .svn)
• Build artifacts (dist/, build/, *.egg-info)
• Development caches (.pytest_cache, .mypy_cache, .ruff_cache)
• IDE files (.vscode, .idea, .DS_Store)
• Test directories (tests/, test/)

Example project structure:

my_package/
├── __init__.py          # Python wrapper
├── my_package_lib.so    # Compiled Nim extension (generated)
├── config.json          # ✅ Included in wheel
├── data.csv             # ✅ Included in wheel
├── models/
│   └── model.bin        # ✅ Included in wheel
└── templates/
    └── template.html    # ✅ Included in wheel

Fine-Grained Control with MANIFEST.in

For precise control over what's included in your wheel, create a MANIFEST.in file in your project root:

include package/config.json
recursive-include package/templates *.html *.css *.js
global-include *.md
exclude package/dev_config.yaml
recursive-exclude package/tests *.py

Supported commands:

• include pattern ... - Include files matching patterns
• exclude pattern ... - Exclude files matching patterns
• recursive-include dir pattern ... - Include files in directory matching patterns
• recursive-exclude dir pattern ... - Exclude files in directory matching patterns
• global-include pattern ... - Include files anywhere matching patterns
• global-exclude pattern ... - Exclude files anywhere matching patterns

Common examples:

# Include specific config file
include package/production.json

# Exclude development configs
exclude package/*-dev.yaml
exclude package/config.local.*

# Include all templates
recursive-include package/templates *

# Include but exclude test data
recursive-include package/data *.csv
recursive-exclude package/data test-*.csv

# Include documentation
global-include *.md
global-include LICENSE

Note: If MANIFEST.in doesn't exist, all package data is included automatically (smart defaults).

CLI Commands

nuwa new <path>

Create a new project scaffold:

nuwa new my_project
nuwa new my_project --name custom-name

nuwa init [path]

Initialize Nuwa in an existing project:

# Initialize in current directory
nuwa init

# Initialize in specific directory
nuwa init /path/to/project

This command:

• Adds [build-system] and [tool.nuwa] to pyproject.toml (or creates it if missing)
• Creates nim/ directory with entry point and helper files
• Updates .gitignore with build artifacts
• Non-destructive: won't overwrite existing files

Use this when you have an existing Python project and want to add Nim extensions to it.

nuwa develop

Compile the project in-place:

# Debug build
nuwa develop

# Release build
nuwa develop --release

# Use build profile
nuwa develop --profile dev
nuwa develop --profile release

# Override configuration
nuwa develop --module-name my_module
nuwa develop --nim-source my_nim_dir
nuwa develop --entry-point main.nim
nuwa develop --output-dir build/
nuwa develop --nim-flag="-d:danger" --nim-flag="--opt:size"

Note: After running nuwa develop, the compiled extension will be in {module_name}/. You can then run python example.py or pytest directly without any installation step.

nuwa watch

Watch for file changes and automatically recompile:

Install the optional watcher first with pip install "nuwa-build[watch]".

# Watch for changes and auto-recompile
nuwa watch

# Watch with tests after each compile
nuwa watch --run-tests

# Watch in release mode
nuwa watch --release

# Watch with build profile
nuwa watch --profile bench

# Override configuration
nuwa watch --module-name my_module
nuwa watch --nim-source my_nim_dir

nuwa build

Build a wheel package for distribution:

# Build a wheel (output: dist/)
nuwa build

# Build with custom Nim flags
nuwa build --nim-flag="-d:release" --nim-flag="--opt:speed"

# Build with profile
nuwa build --profile release

# Override configuration
nuwa build --module-name my_module
nuwa build --nim-source my_nim_dir
nuwa build --entry-point main.nim
nuwa build --output-dir build/

Note: The nuwa build command creates wheels in the dist/ directory, following standard Python packaging conventions. This is the recommended way to build wheels for distribution instead of using pip wheel or python -m build.

nuwa clean

Clean build artifacts and dependencies:

# Clean everything (dependencies + artifacts)
nuwa clean

# Clean only dependencies (.nimble/ directory)
nuwa clean --deps

# Clean only build artifacts and cache
nuwa clean --artifacts

What gets cleaned:

• --deps: Removes the .nimble/ directory (local Nimble packages)
• --artifacts: Removes nimcache/, build/, dist/ directories and compiled .so/.pyd files
• (no flags): Cleans both dependencies and artifacts

Entry Point Discovery

If entry-point is not specified, Nuwa will automatically discover the main entry point using this priority:

1. Explicit [tool.nuwa] entry-point configuration
2. {module_name}_lib.nim (matches the lib-name)
3. lib.nim (fallback convention)
4. First (and only) .nim file if only one exists
5. Error if multiple files found and no clear entry point

Mixing Python and Nim

Your __init__.py can import from the compiled Nim extension and add Python wrappers:

# In my_package/__init__.py
from .my_package_lib import *

__version__ = "0.1.0"

# Example: Wrap Nim functions with Python code
def validate_dataframe(df, column_name):
    """Extract pandas data and pass to Nim for zero-copy validation"""
    import numpy as np
    from ctypes import c_void_p

    # Extract data as numpy array (zero-copy view)
    data = df[column_name].to_numpy()

    # Get pointer and pass to Nim for validation
    result = validate_array_raw(
        data.ctypes.data_as(c_void_p),
        len(data)
    )
    return result

This allows you to:

• Use Python to extract/prepare data (e.g., from pandas DataFrames)
• Pass pointers/arrays to Nim for zero-copy processing
• Return results back to Python for formatting

Multi-File Projects

Nim's module system handles dependencies automatically. Use include to add code from other Nim files:

nim/my_package_lib.nim:

import nuwa_sdk  # Provides nuwa_export for automatic type stub generation
include helpers  # Include helpers.nim from same directory

proc greet(name: string): string {.nuwa_export.} =
  return make_greeting(name)

proc add(a: int, b: int): int {.nuwa_export.} =
  return a + b

nim/helpers.nim:

proc make_greeting(name: string): string =
  return "Hello, " & name & "!"

Compile my_package_lib.nim and both modules are included in the final .so/.pyd file.

Important: Use include (not import) when building shared libraries. The include directive literally includes the code at compile time, while import creates a separate module namespace.

Exporting Functions to Python

You must add the {.nuwa_export.} pragma to any Nim procedure you want to access from Python.

• ✅ Exported: proc add(a: int, b: int): int {.nuwa_export.} - Accessible from Python
• ❌ Not exported: proc add(a: int, b: int): int - Not accessible from Python

The {.nuwa_export.} pragma does two things:

1. Makes the function callable from Python (via the underlying nimpy library)
2. Generates type stub information for IDE autocomplete and type checking

Common mistake: Forgetting the pragma means your function won't be available in Python, even though it compiles successfully.

Output Location

The output-location setting controls where the compiled extension is placed:

• "auto" (default): Uses flat layout - places extension in {module_name}/

• "src": Explicitly uses src/{module_name}/ (for compatibility with old projects)

• Explicit path: Use a custom output directory

Python Usage

Once compiled and installed, use your Nim extension like any Python module:

import my_package

# Call Nim-compiled functions (imported via __init__.py)
result = my_package.greet("World")
print(result)  # "Hello, World!"

sum_result = my_package.add(5, 10)
print(sum_result)  # 15

Contributing

Contributions are welcome! The codebase is well-organized with:

• Full type hints and mypy validation
• Comprehensive error handling
• Proper logging support
• Context managers for resource management
• Clear module boundaries and separation of concerns

Troubleshooting

"Nim compiler not found"

Make sure Nim is installed and in your PATH:

nim --version

Install from https://nim-lang.org/install.html if needed.

"cannot open file: nimpy"

You need to install the nimpy library. You can either:

Option 1: Auto-install via configuration (Recommended)

[tool.nuwa]
nimble-deps = ["nimpy"]

Option 2: Manual installation

nimble install nimpy

"nimble package manager not found"

Nimble is installed with Nim. Make sure Nim is properly installed and in your PATH:

nim --version
nimble --version

If nimble is not found, reinstall Nim from https://nim-lang.org/install.html.

"ModuleNotFoundError: No module named 'my_package'"

The module needs to be compiled first. Run:

nuwa develop

Then you can import it directly from the project root. No pip install needed!

For pytest: Make sure you've compiled the extension with nuwa develop first. The flat layout allows pytest to discover the module automatically.

"ValueError: Module name '...' is not a valid Python identifier"

Your project name contains invalid characters for Python modules. Module names can only contain letters, numbers, and underscores, and cannot start with a number. Use the --name option:

nuwa new my-project --name my_valid_name

"Multiple .nim files found in nim/"

Nuwa found multiple .nim files but can't determine which is the entry point. Specify it in pyproject.toml:

[tool.nuwa]
entry-point = "my_entry_file.nim"

Or ensure there's only one .nim file, or name your entry point {module_name}_lib.nim.

License

MIT

Acknowledgments

• Uses nimpy for Python bindings
• Named after Nüwa, the Chinese goddess of creation