XTC

Links

Refer to documentation at https://xtc-tools.github.io/xtc

Refer to tutorials here and to additionnal developers documentation here.

Overview

XTC is a domain-specific dataflow graph compiler for linear algebra operations. It provides:

• Operational DSL: Define computation graphs with tensors and operators
• Scheduling DSL: High-level transformations (tiling, parallelization, vectorization, etc.)
• Multiple backends: MLIR (linalg + transform), TVM (Tensor IR), JIR (INRIA internal)
• Autotuning: Definition and exploration of the optimization space

Build & Development

Installation

If needed, install uv following the instructions here.

Debian-like x86_64 or aarch64 Linux distributions (Python: 3.10 to 3.14 inclusive):

sudo apt install python3 python3-dev build-essential libomp5 binutils binutils-aarch64-linux-gnu binutils-x86-64-linux-gnu
sudo apt install libpfm4-dev # Optional: interface to Linux perf counters
sudo sysctl kernel.perf_event_paranoid=1 # Optional: give access to hardware counters
uv venv -p 3.12 && source .venv/bin/activate
uv pip install -e '.[dev]'
make test

MacOs M1+ macos-14/macos-15 (Python: 3.10 to 3.14 inclusive):

brew install libomp x86_64-linux-gnu-binutils aarch64-elf-binutils
export DYLD_LIBRARY_PATH="/opt/homebrew/opt/libomp/lib:$DYLD_LIBRARY_PATH"
uv venv -p 3.12 && source .venv/bin/activate
uv pip install -e '.[dev]'
make test

Note that [dev] extension installs both mlir and tvm backends in addition to development tools.

Available extensions are:

• [mlir]: MLIR backend
• [tvm]: TVM backend
• [test]: develop, docs and tests tools
• [default]: mlir + tvm
• [dev]: default + test

Using a local LLVM / MLIR build

By default XTC uses the LLVM/MLIR toolchain shipped in its Python wheels. To use a local LLVM checkout instead (e.g. to test compiler changes), point XTC at your build directory ($LLVM_BUILD below). Two levels are possible.

Binaries only. Override opt, llc, mlir-opt and mlir-translate while keeping the wheel's Python bindings — the simplest option, e.g. when your changes live in the LLVM middle-end / back-end:

export XTC_LLVM_PREFIX=$LLVM_BUILD   # providing bin/opt and bin/llc
export XTC_MLIR_PREFIX=$LLVM_BUILD   # providing bin/mlir-opt and bin/mlir-translate

Binaries and Python bindings. The MLIR bindings are a native extension, so the interpreter must match the Python version they were built for (check the ABI tag under $LLVM_BUILD/tools/mlir/python_packages/mlir_core/mlir/_mlir_libs). Create a matching venv if, install XTC without the MLIR wheels, build the runtime support libraries, then prepend the bindings to PYTHONPATH:

uv venv -p 3.14 .venv-local && source .venv-local/bin/activate
uv pip install -e '.[tvm]'

export PYTHONPATH=$LLVM_BUILD/tools/mlir/python_packages/mlir_core:$PYTHONPATH
export XTC_MLIR_PREFIX=$LLVM_BUILD
export XTC_LLVM_PREFIX=$LLVM_BUILD
export XTC_MLIR_TARGET=llvmir

Code quality

Code quality requirements:

• Type annotations: Strict pyright mode, full annotations required
• Formatting: Ruff (line length 88)
• License headers: BSD-3-Clause required on all source files
• All checks must pass before merge

Type checking:

make check-type          # Run both pyright and mypy
pyright                  # Run pyright only
mypy                     # Run mypy only

Formatting:

make format              # Apply all formatting (license + ruff)
make check-format        # Check formatting without modifying files

Testing structure:

• tests/pytest/unit/: Core interface unit tests
• tests/pytest/{mlir,tvm}/: Backend-specific tests
• tests/filecheck/: Lit+FileCheck functional tests for code generation

Global test commands:

make test                # Run minimal unit tests
make check               # Run ALL acceptance tests (required for contributions)
make check-pytest        # Run pytest suite only
make check-lit           # Run LIT tests for LLVM IR target
make check-lit-c         # Run LIT tests for C target
pytest tests/pytest/unit # Run specific test directory

Running individual tests:

# Single pytest file
pytest tests/pytest/unit/test_specific.py -v

# Single lit test
lit -v tests/filecheck/backends/specific_test.py

# C target for lit tests
XTC_MLIR_TARGET=c lit -v tests/filecheck/backends/specific_test.py

Dependencies update

Python package dependencies are listed in dependencies.toml with definition of groups and groups dependencies.

Always update dependencies there and run make dependencies to update pyproject.toml before commit.

Releases

Package versions are derived from Git tags. Successful updates to main are published to TestPyPI as development versions, while tags of the form xtc-vX.Y.Z are published to PyPI. See the release guide for the release procedure and required trusted-publishing configuration.

Architecture

Core Abstractions (src/xtc/itf/)

Abstract interfaces defining the compilation pipeline:

• data/ - Tensor, DataType, ShapeType
• operator/ - Linear algebra operator interface
• graph/ - Graph, Node, Operation abstractions
• back/ - Backend interface
• schd/ - Scheduler and Schedule abstractions
• comp/ - Compiler interface
• exec/ - Executor and Evaluator interfaces
• search/ - Search space exploration interface

Backends (src/xtc/backends/)

Exposed backends:

• mlir/ - MLIR backend using linalg + transform dialects
• tvm/ - TVM backend using Tensor IR + Schedule APIs

XTC also supports multiple MLIR Targets for the code generation:

• llvmir (default)
• c
• nvgpu

To force the use of a specific target, you can set the env variable XTC_MLIR_TARGET=<mlir-target>.

The MLIR backend can be extended using the SDist extension, which provides distribution primitives. To install SDist, follow the instructions in docs/develop/optional_backends.md, in the "MLIR development version" section.

Note that the nvgpu target requires a recent version of Cuda (tested with Cuda 13.0). By default, it tries to find Cuda at /usr/local/cuda, but it can be overridden with the CUDA_INSTALL_DIR env variable. The performance counters can be accessed is the GPU has a compute capability >=7.5.

Compilation Pipeline

1. User defines Graph with Tensors and Operators
2. Backend created from Graph
3. Scheduler applies transformations and produces Schedule
4. Compiler generates executable Module
5. Executor/Evaluator runs and measures performance

CLI Tools (src/xtc/cli/)

• mlir-loop - High-level scheduling for MLIR linalg operators
• mlir-backend - MLIR backend wrapper
• loop-explore - Autotuning and space exploration
• loop-display - Visualization of exploration results

AI assistants

To create agent guidance files from this README: make agents (AGENTS.md) or make claude (CLAUDE.md)