Talyn: Robust, Stable AsyncIO Event Loop for Python

Talyn is a robust, exceptionally stable, and realistically fast asyncio event loop drop-in replacement for Python, powered by the asynchronous capabilities of Zig and io_uring.

Talyn prioritizes correctness, complete system safety, and high usability over artificial micro-benchmark superiority. It is fully compatible with CPython's standard single-threaded and free-threaded (GIL-disabled) runtimes.

🚀 Features

• Realistic Speed: Designed to deliver solid and reliable I/O performance on Linux by leveraging io_uring's native kernel-side asynchronous completion queues.
• Robust & Crash-Resistant: Meticulously hardened against circular reference memory leaks, stack alignment faults, signal interrupt deadlocks, and use-after-free bugs (332 bugs identified and resolved across audit passes with 0 open bugs).
• Full Asyncio Compatibility: Passes 100% of standard Python asyncio, subprocess, transports, and connection-lifecycle test suites.
• Offline AST Bug Hunter & Linter: Enforces strict invariant safety rules (preventing memory leaks, discarded syscalls, panics, and UAFs) via a built-in, sub-15ms Zig and Python AST static analyzer (zig build lint).
• Open Knowledge Documentation: Built with comprehensive architectural mandates, lessons learned, and bug tracking under Google Open Knowledge Format (OKF v0.2).
• Modern Packaging: Fully migrated to PEP 517/518 standard declarative pyproject.toml configuration.
• GIL-Disabled Free-Threading Ready: Fully compatible with python3.13t and python3.14t without memory races.

📜 Requirements

• Python: >= 3.13 (Tested and verified under CPython 3.13, 3.14, 3.13t (free-threaded), and 3.14t (free-threaded))
• Linux Kernel: >= 7.0 (Verified on Linux Kernel 7.0.x)
• Zig Compiler (for source builds): 0.16.0 or 0.17.0 (Fedora packages / upstream release)

🐳 io_uring in Containers and Sandboxes

Talyn needs working io_uring syscalls (io_uring_setup, io_uring_enter, io_uring_register). A recent kernel alone is not enough: many container runtimes and sandboxes filter these syscalls out. If Talyn fails at startup with a permission error during ring creation, check which case you're in:

• EPERM from io_uring_setup — blocked by a seccomp filter or by the kernel.io_uring_disabled sysctl. Tell them apart with sysctl kernel.io_uring_disabled: 1 means the sysctl is the cause (io_uring then requires CAP_SYS_ADMIN), 0 means seccomp is the cause.
• ENOSYS — the kernel has no io_uring at all, or the sandbox doesn't emulate it. Talyn cannot run there; no workaround.

Background: a wave of io_uring kernel CVEs in 2023–2024 led Docker (v25.0) and containerd (v2.0) to drop the io_uring syscalls from their default seccomp allowlists. This is the runtime's security policy, not a Talyn defect — the same binary runs fine on any normal VM or bare-metal host with a recent kernel. See docs/development/containers-and-sandboxes.md for full details.

🔧 Installation & Testing

To compile and install Talyn locally on a native Linux machine, run:

pip install -e .

🍎 macOS Development & Testing via Podman (Apple Silicon)

Since Talyn is a Linux-only project leveraging io_uring, development and testing on macOS require running inside a Linux environment.

We provide a frictionless, automated setup using Podman and a Fedora 44 (AARCH64) container:

1. Install Podman:

   brew install podman
   

2. Initialize the Podman VM (only needed once):

   podman machine init
   

3. Run the test suite (the script automatically starts the VM if it's stopped, builds the image, runs the tests with proper permissions, and stops the VM at the end if it started it):

   ./scripts/macos/run_tests.sh
   

You can pass any options supported by test_all.sh (e.g., --verbose or --python=3.13):

./scripts/macos/run_tests.sh --verbose --python=3.13

To force a rebuild of the Fedora testing image:

./scripts/macos/run_tests.sh --rebuild-image

📊 Benchmarking

We also provide a wrapper to run the benchmark suite inside the Fedora container using the optimized ReleaseFast target:

./scripts/macos/benchmark.sh --python=python3.13

You can target a specific benchmark using --bench (note that benchmark names with spaces must be quoted):

./scripts/macos/benchmark.sh --python=python3.13 --bench="task spawn"

The generated comparison plots will be automatically saved to benchmarks/output/.

📦 Building & Publishing Multi-Architecture Wheels

If you are developing on macOS (Apple Silicon) and need to build and publish wheels for both aarch64 and x86_64 architectures to PyPI, you can do so in a single command using Podman's emulation:

1. Build all wheels: This script builds a native aarch64 container image and an emulated x86_64 container image, runs scripts/build.sh inside both, and collects all 8 wheels (4 Python versions × 2 architectures) in the ./dist/ directory:

   ./scripts/macos/build_all_wheels.sh
   

2. Publish to PyPI: Upload the built distributions in ./dist/ using twine (pre-configured in your repository):

   ./scripts/publish.sh
   

🖥️ Linux (x86_64) Development: Build, Cross-Compile & Multi-Arch Testing

All of Talyn's development—including aarch64 and riscv64 wheels and full test-suite runs—can be done on an x86_64 (Intel) Linux PC, with no MacBook required:

• Building: Zig is a native cross-compiler, so foreign-architecture wheels and the extension are produced at full host speed — no QEMU involved.
• Testing: QEMU user-mode emulation (e.g. podman run --platform linux/arm64) cannot run Talyn, because io_uring_setup returns ENOSYS under user-mode emulation and Talyn requires io_uring with no fallback. Instead, we boot a full-system Fedora 44 VM per architecture, whose real guest kernel passes io_uring syscalls through to the host kernel.

📋 1. Fedora 44 required packages

Install Zig, all four Python interpreters with development headers, and the stdlib test suites:

sudo dnf install zig \
    python3.13 python3.13-devel python3.13-freethreading \
    python3.14 python3.14-devel python3.14-freethreading python3.14-freethreading-devel \
    python3.13-test python3.14-test python3.14-freethreading-test

Bootstrap pip and the test tooling for each interpreter:

for p in python3.13 python3.13t python3.14 python3.14t; do
    $p -m ensurepip --upgrade
    $p -m pip install --user pytest pytest-asyncio
done

For foreign-architecture VM testing (aarch64 / riscv64), additionally:

sudo dnf install qemu-system-aarch64 edk2-aarch64 qemu-system-riscv edk2-riscv64 \
    genisoimage openssh-clients

🏗️ 2. Build all wheels natively (x86_64 + aarch64 + riscv64)

./scripts/linux/build_all_wheels.sh

Uses Zig's native cross-compiler — no QEMU needed. Produces 12 wheels (4 Python variants × 3 architectures) in ./dist/, ready for ./scripts/publish.sh.

🧪 3. Run the test suite natively (x86_64)

./scripts/test_all.sh                  # Debug build
./scripts/test_all.sh --starburst      # ReleaseFast build

🖥️ 4. Test foreign-architecture wheels in VMs

Boots a real Fedora 44 VM per architecture (first run downloads the ~0.5 GB cloud image to ~/.cache/talyn-*-vm/) and runs the full pytest suite against each installed wheel:

./scripts/linux/run_tests.sh                     # aarch64 wheels
./scripts/linux/run_tests.sh --arch=riscv64      # riscv64 wheels
./scripts/linux/run_tests.sh --smoke             # quick import + event-loop smoke test only
./scripts/linux/run_tests.sh --shutdown          # stop the VM after the run

⚡ 5. Run the full test_all.sh suite in VMs (cross-compiled, fast)

test_all.sh normally compiles the extension inside the VM, which is slow under QEMU TCG. Instead, cross-compile the four extension variants natively on the host (the same build flags the wheels use) and run test_all.sh --no-build in the VM:

./scripts/linux/run_test_all.sh --arch=aarch64 --starburst
./scripts/linux/run_test_all.sh --arch=riscv64 --starburst

Under heavy emulation, a stdlib module may exceed its per-module timeout (e.g. test_subprocess on riscv64); raise it with TALYN_STDLIB_TIMEOUT=600 and the memory-safety repro timeout with TALYN_REPRO_TIMEOUT=900.

🛡️ 6. Run the native Offline AST Linter & Bug Hunter

Talyn includes a zero-dependency static analyzer hooked into std.zig.Ast and Python's ast module to prevent regressions and enforce architectural safety rules in under 15ms:

zig build lint

For the complete catalog of rules, see docs/development/ast-linter.md.

⏱️ Measured timings

scripts/test_all.sh --starburst (4 Python variants: pytest suite + stdlib asyncio suite):

All configurations pass the pytest suite; under QEMU TCG the emulated runs are ~5× slower than native, and the cross-built runs cut the in-VM build phase (~2.4× speedup on aarch64). An occasional stdlib-asyncio module may time out under heavy emulation (see note above about TALYN_STDLIB_TIMEOUT).

⚡ Optimization & Target Compilation (For Developers & Power Users)

By default, the pre-built wheels generated by build.sh are compiled targeting a generic x86_64 CPU architecture baseline to ensure 100% universal compatibility across all 64-bit modern x86 Linux processors (e.g., matching standard PyPI manylinux wheel compatibility).

Based on our benchmarks and comprehensive validation—including 100% passing results in the standard asyncio test suite across four distinct Python versions—we publish the official binary packages (wheels) compiled in Starburst mode (Zig built with ReleaseFast) to deliver peak performance out of the box with proven stability.

If you are compiling from source, you can customize the compilation optimize mode and target CPU architecture to unleash maximum performance:

1. Compile and Optimize for your Native CPU (Highly Recommended)

To compile Talyn so that it takes full advantage of your host's exact CPU instructions (such as AVX2, AVX-512, cache alignment, etc.):

# Omit TALYN_CPU so Zig defaults to native CPU optimization
TALYN_OPTIMIZE=ReleaseFast pip install .

2. Configure Compilation Modes

You can control Zig's optimize mode by setting the TALYN_OPTIMIZE environment variable (defaults to Debug for developer convenience):

• TALYN_OPTIMIZE=Debug (Default): Compiles with heavy runtime assertions and debug symbols.
• TALYN_OPTIMIZE=ReleaseSafe: Compiles with full optimizations but keeps safety checks (e.g. out-of-bounds, overflows).
• TALYN_OPTIMIZE=ReleaseFast: Compiles with maximum optimizations (Starburst mode), disabling safety checks for peak execution speed.

3. Target a Specific CPU microarchitecture

You can force Zig to compile for a specific target CPU microarchitecture (like x86_64_v2 or x86_64_v3):

TALYN_OPTIMIZE=ReleaseFast TALYN_CPU=x86_64_v3 pip install .

4. Zig Version Compatibility

Talyn builds cleanly on both Zig 0.16.0 and Zig 0.17.0 from a single source tree.

All unit tests and full test suites pass under both toolchains:

# Zig 0.16.0
zig build test
./scripts/test_all.sh --starburst --verbose

# Zig 0.17.0
PATH=/path/to/zig-0.17.0:$PATH zig build test
PATH=/path/to/zig-0.17.0:$PATH ./scripts/test_all.sh --starburst --verbose

📦 Usage

Basic Usage

To run a coroutine directly with the Talyn event loop:

import talyn
import asyncio

async def main():
    print("Hello from Talyn!")
    await asyncio.sleep(1)
    print("Goodbye from Talyn!")

# Run using Talyn event loop
talyn.run(main())

Graceful Fallback & Hardening Check

For production configurations, you may want to support fallback options. If Talyn is not installed, or if the host Linux kernel does not meet Talyn's safety baseline (monitored by the HARD-01 Kernel Version Guard which requires kernel >= 6.0), the code will gracefully fall back to uvloop (if available), and finally to standard Python asyncio:

import asyncio

# 1. Try to initialize Talyn (performs HARD-01 kernel version check)
try:
    import talyn
    talyn.install()
except (ImportError, RuntimeError):
    # 2. Fallback to uvloop if Talyn is missing or kernel version is too old
    try:
        import uvloop
        uvloop.install()
    except (ImportError, AttributeError):
        # 3. Fallback to standard Python asyncio (do nothing)
        pass

async def main():
    print("Running with the best available event loop!")

asyncio.run(main())

💝 Historical Credits & Origin

Talyn is spun off from Leviathan, an event loop originally pioneered by Enrique Mora. Enrique Mora's creative spark and vision of merging Zig, io_uring, and asyncio laid the critical foundation and architecture of this project.

As Talyn evolved, the implementation underwent a complete systems-level refactoring to transition from a theoretical prototype to a production-grade, crash-resistant runtime:

• Eliminated multi-crossing Zig/Python vectorcall overhead by implementing a fused scheduler step trampoline in pure Zig.
• Redesigned completion handlers into flat, GC-safe ring buffers.
• Fully audited and resolved all memory-leak reference cycles under concurrent connections.

To honor the project's roots and Enrique's early work:

• The original Leviathan README can be viewed at: docs/historical/leviathan-readme.md
• The original preliminary benchmarks can be viewed at: docs/historical/leviathan-benchmark.md

📖 Project Story & Documentation

• Master Documentation Index — Architectural mandates, bug tracker (332 resolved bugs), and lessons learned under OKF v0.2.
• Development Journey — The full story: from discovery to challenges, the shift from "ultra-fast" to "realistic fast and stable", and how Talyn was built.
• Why Talyn? — The personal story and meaning behind the new name.

📄 License

This project is licensed under the MIT License. See LICENSE.md for details.