Skip to main content

TVM-FFI OrcJIT

A Python package that enables dynamic loading of compiled object files (.o) using LLVM ORC JIT v2, providing a flexible JIT execution environment for TVM-FFI exported functions.

Features

  • JIT Execution: Load and execute compiled object files at runtime using LLVM's ORC JIT v2
  • High-Level Loading: default_session().load_module(...) mirrors tvm_ffi.load_module, returning a plain tvm_ffi.Module
  • Unified Input: Load from a file path, in-memory object bytes, or a list mixing both
  • Shared Session: A process-wide session so callers share process-symbol resolution and the Linux slab pool while loaded modules remain isolated
  • Symbol Isolation: Separate load_module calls define independent symbol namespaces, so they can define the same symbol without conflicts
  • Bounded-Range Memory: A growable Linux slab pool keeps JIT code/data close enough for 32-bit PC-relative relocations and supports explicit drained-slab reclamation
  • Init/Fini Support: Handles static constructors/destructors across ELF (.init_array/.ctors), Mach-O (__mod_init_func), and COFF (.CRT$XC*/.CRT$XT*)
  • Cross-Platform: Linux (x86_64, aarch64), macOS (arm64), Windows (AMD64, ARM64)
  • Multi-Compiler: Tested with LLVM Clang, GCC, Apple Clang, MSVC, and clang-cl
  • TVM-FFI Integration: Seamlessly works with TVM-FFI's stable C ABI
  • Python API: Simple Pythonic interface for JIT compilation and execution

Supported Platforms and Compilers

Object files compiled with any of the following compiler/platform combinations can be loaded and executed by the ORC JIT:

Platform Compilers C C++
Linux (x86_64, aarch64) LLVM Clang, GCC yes yes
macOS (arm64) LLVM Clang, Apple Clang yes yes
Windows (AMD64, ARM64) LLVM Clang, MSVC, clang-cl yes no

Windows is C-only across all compilers. C++ objects compiled with TVM_FFI_DLL_EXPORT_TYPED_FUNC use try/catch (via TVM_FFI_SAFE_CALL_BEGIN/END), which requires Itanium exception ABI symbols (__cxa_begin_catch, __gxx_personality_v0, etc.) that the MSVC-built host process cannot provide. Pure C objects using the TVMFFISafeCallType ABI work on all platforms.

Windows ARM64 uses LLVM's RTDyldObjectLinkingLayer because LLVM 23.1.1 does not implement COFF/AArch64 in JITLink. RuntimeDyld handles ARM64 relocations, long-branch stubs, and DLL-import slots; the addon preserves .CRT$XC* / .CRT$XT* constructor/destructor handling. Windows AMD64 continues to use JITLink.

Installation

Install from PyPI

pip install apache-tvm-ffi apache-tvm_ffi_orcjit

Build from Source

Prerequisites

  • Python 3.10+, CMake 3.24+, C++17 compiler

The build automatically installs mlc-llvm-static from PyPI. It supplies LLVM 23.1.1, static zlib/zstd, and the Linux ORC runtime. CMake uses the SDK in the build environment and links its exported static targets. A separate LLVM installation is not used. The SDK is linked statically and is not a runtime dependency of the resulting addon wheel.

The base SDK contains the native backend used by ORC JIT. mlc-llvm-static-extra-targets adds cross-compilation backends and is not required for this addon. Clang and other compilers are separate tools; the SDK does not include them.

Build and install

git clone --recursive https://github.com/apache/tvm-ffi.git
cd tvm-ffi

# Install tvm-ffi first
pip install -e .

# Build and install the orcjit addon
cd addons/tvm_ffi_orcjit
pip install -e .

For a build with --no-build-isolation, install the build requirements, including mlc-llvm-static==23.1.1.post0, into the same Python environment first. LLVM_PREFIX and LLVM_DIR do not select a different SDK.

On Linux, each load_module call follows the same default compiler selection as tvm_ffi.cpp.build: $CXX, or c++ when unset. ORC asks that compiler for its libstdc++.so.6 and optional libstdc++_nonshared.a, then keeps both local to the new JITDylib. Discovery is cached, and load_module(..., cxx="g++-14") can select a different toolchain without supplying runtime-library paths.

Usage

Basic Example

The high-level API mirrors tvm_ffi.load_module: a process-wide shared session plus a load_module that accepts a path, in-memory object bytes, or a list of either, and returns a plain tvm_ffi.Module.

import tvm_ffi_orcjit as oj

# Shared process-wide session (created once, cached).
session = oj.default_session()

# Load a single object file by path.
mod = session.load_module("example.o")

# Call an exported function.
result = mod.add(1, 2)
print(f"Result: {result}")  # Output: Result: 3

Loading Multiple Objects and In-Memory Bytes

Objects passed together are linked into one module (the same way a multi-object shared library links). Each element may be a path or an object-file image in memory.

from pathlib import Path

session = oj.default_session()

mod = session.load_module(
    [
        "math_ops.o",                    # path
        Path("kernel.o").read_bytes(),   # in-memory object bytes
    ]
)
result = mod.call_math(10, 20)

Isolated Sessions

default_session() is shared across the process. For an isolated symbol namespace or a tuned memory arena, construct an ExecutionSession directly:

from tvm_ffi_orcjit import ExecutionSession

session = ExecutionSession()          # independent LLVM ExecutionSession
mod = session.load_module("impl.o")

Writing Functions for OrcJIT

C++ (Linux/macOS)

#include <tvm/ffi/function.h>

TVM_FFI_DLL_EXPORT_TYPED_FUNC(add, [](int a, int b) {
    return a + b;
});

Compile: clang++ -std=c++17 -fPIC -O2 -c -o example.o example.cc

Pure C (all platforms including Windows)

#include <tvm/ffi/c_api.h>

TVM_FFI_DLL_EXPORT int __tvm_ffi_add(
    void* self, const TVMFFIAny* args, int32_t num_args, TVMFFIAny* result) {
  result->type_index = kTVMFFIInt;
  result->v_int64 = args[0].v_int64 + args[1].v_int64;
  return 0;
}

Compile: clang -O2 -c -o example.o example.c

How It Works

  • LLJIT: Built on LLVM's ORC JIT v2 with ObjectLinkingLayer (JITLink) for all platforms.
  • ELF Lifecycle (Linux): LLVM 23.1.1+ ELFNixPlatform runs ordered constructors and destructors through LLJIT::initialize / deinitialize.
  • InitFiniPlugin (macOS/Windows): A local ObjectLinkingLayer::Plugin handles Mach-O __mod_init_func/__mod_term_func and COFF .CRT$XC*/.CRT$XT* while those ORC platforms remain unsuitable.
  • DLL Import Stubs (Windows): Custom DefinitionGenerator that resolves host process symbols from all loaded DLLs and creates __imp_* pointer stubs in JIT memory, keeping all fixups within PCRel32 range.
  • SEH Stripping (Windows): ObjectTransformLayer strips .pdata/.xdata relocations from COFF objects before JITLink graph building, working around a JITLink limitation with COMDAT section symbols.

Refer to ORCJIT_PRIMER.md for background on object files, linking, LLVM ORC JIT v2, and the addon's architecture.

Project Structure

tvm_ffi_orcjit/
├── CMakeLists.txt              # Build configuration
├── pyproject.toml              # Python package metadata
├── src/ffi/
│   ├── orcjit_session.cc       # ExecutionSession (LLJIT setup, plugins)
│   ├── orcjit_session.h
│   ├── orcjit_dylib.cc         # JIT dylib module (object loading, symbol lookup)
│   ├── orcjit_dylib.h
│   ├── orcjit_memory_manager.*  # Growable Linux slab pool
│   ├── orcjit_slab.*            # Contiguous-VA allocator and page protection
│   ├── llvm_patches/            # Isolated upstream LLVM workarounds
│   └── orcjit_utils.h          # LLVM error handling utilities
├── python/tvm_ffi_orcjit/
│   ├── __init__.py             # Module exports and library loading
│   └── session.py              # Python ExecutionSession + default_session
├── tests/                      # See tests/README.md
└── examples/quick-start/       # Complete example with CMake

CI

Runs on Linux (x86_64, aarch64), macOS (arm64), Windows (AMD64, ARM64) via cibuildwheel. Each platform builds test objects with multiple compilers and runs the full test suite. See the orcjit job in .github/workflows/ci_test.yml.

Troubleshooting

"Cannot find global function" error

The shared library wasn't loaded. Reinstall: pip install --force-reinstall apache-tvm_ffi_orcjit

"Duplicate definition of symbol" error

Use separate libraries for different implementations of the same symbol.

"Symbol not found" error

Ensure functions are exported with TVM-FFI macros (TVM_FFI_DLL_EXPORT_TYPED_FUNC for C++, or __tvm_ffi_ prefix for C).

Relocation errors on Windows

MSVC/clang-cl objects must be compiled with /GS- to disable buffer security checks (__security_cookie) which are CRT symbols the JIT cannot resolve.

LLVM SDK discovery failure

Source builds require the pinned mlc-llvm-static package in the build interpreter's environment. Normal isolated builds install it automatically. For builds with --no-build-isolation, install it explicitly:

pip install mlc-llvm-static==23.1.1

Reclaiming unused JIT memory on Linux

Dropping a module returns its regions to the session's slab pool for reuse. To return fully drained slabs to the operating system, call session.clear_free_slabs(). The call is synchronized with concurrent JIT work.

License

Apache License 2.0

Metadata

Release files for apache-tvm-ffi-orcjit 0.1.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for apache-tvm-ffi-orcjit 0.1.3
File Actions
apache_tvm_ffi_orcjit-0.1.3-py3-none-win_arm64.whl Python 3 none Windows ARM64 Details for apache_tvm_ffi_orcjit-0.1.3-py3-none-win_arm64.whl
apache_tvm_ffi_orcjit-0.1.3-py3-none-win_amd64.whl Python 3 none Windows x86-64 Details for apache_tvm_ffi_orcjit-0.1.3-py3-none-win_amd64.whl
apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl Python 3 none Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl Python 3 none Linux glibc 2.26+ ARM64, Linux glibc 2.28+ ARM64 Details for apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
apache_tvm_ffi_orcjit-0.1.3-py3-none-macosx_14_0_arm64.whl Python 3 none macOS 14.0+ ARM64 Details for apache_tvm_ffi_orcjit-0.1.3-py3-none-macosx_14_0_arm64.whl

Total release size: 68.2 MB

Release files / apache_tvm_ffi_orcjit-0.1.3-py3-none-win_arm64.whl

Download URL apache_tvm_ffi_orcjit-0.1.3-py3-none-win_arm64.whl
Size 11.2 MB
Tags Python 3 Windows ARM64
SHA-256 checksum
How to use checksums
2568daf3f7c646ccba319ee17149625c6b61e0057b80ef8746af96e4c46e3d6d
BLAKE2b-256 checksum
How to use checksums
bbfad67ec91bfad9be7f072bd6888ca9d342475d39b028a9de08c311bbcbadd5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / apache_tvm_ffi_orcjit-0.1.3-py3-none-win_amd64.whl

Download URL apache_tvm_ffi_orcjit-0.1.3-py3-none-win_amd64.whl
Size 12.2 MB
Tags Python 3 Windows x86-64
SHA-256 checksum
How to use checksums
7f6633516c39fea047fdd8613e5e281b4e2853867265c68d899b062fa4376551
BLAKE2b-256 checksum
How to use checksums
7dac29bb819c29e36c64a6d3ee8cc42ea0fe89dfda8f093caf1ffee29058f030
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 16.0 MB
Tags Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 Python 3
SHA-256 checksum
How to use checksums
04f0ea19f4ae7c89564202067c34b697ce7afc3bb1158ecbdf7c0d1f4951f156
BLAKE2b-256 checksum
How to use checksums
1ba6514d719264fd723a47fa9c3ddc01ab66e2f626b87d569e396752138537e9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl

Download URL apache_tvm_ffi_orcjit-0.1.3-py3-none-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl
Size 14.9 MB
Tags Linux glibc 2.26+ ARM64 Linux glibc 2.28+ ARM64 Python 3
SHA-256 checksum
How to use checksums
c134044db04e39f01ef904b68b03cbf2596cbee108aebe037454efc709f8f89b
BLAKE2b-256 checksum
How to use checksums
6595757f5ea8c53e8d982f137dba66ba295f5f2085d306130a9e256520bb2b02
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / apache_tvm_ffi_orcjit-0.1.3-py3-none-macosx_14_0_arm64.whl

Download URL apache_tvm_ffi_orcjit-0.1.3-py3-none-macosx_14_0_arm64.whl
Size 14.1 MB
Tags Python 3 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
6a63b081af7ae20e4f064ac1159a406cd665be317bece459c443fd3f6a6949a0
BLAKE2b-256 checksum
How to use checksums
d623768d60e2754399cbf3a226059c4cbaf5c82378aca4b19f3d1c7496bc8551
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Security for apache-tvm-ffi-orcjit 0.1.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 5 files in this release have been uploaded with provenance metadata.

Select from 5 files

Attestation for apache_tvm_ffi_orcjit-0.1.3-py3-none-win_arm64.whl

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page