PyDotCompute

A Python port of DotCompute's Ring Kernel System - a GPU-native actor model with persistent kernels and message passing.

Overview

PyDotCompute brings GPU-native actor model capabilities to Python, enabling developers to create persistent GPU kernels that communicate through message queues. This approach is ideal for:

• Real-time GPU compute pipelines
• Streaming data processing on GPU
• Actor-based GPU programming
• High-throughput message-driven architectures

Performance Highlights

Metric	Value
Message latency (p50)	21μs
Message latency (p99)	131μs
GPU graph processing	1.7M edges/sec
Actor throughput	76K msg/sec
Cython queue ops	0.33μs

Benchmarked with uvloop on Linux. See Benchmarks for details.

Features

• Ring Kernel System: Persistent GPU kernels with infinite loops and message queues
• High Performance: uvloop auto-installation for 21μs message latency
• Message Passing: Type-safe message serialization with msgpack
• Unified Memory: Transparent host-device memory management with lazy synchronization
• Lifecycle Management: Two-phase launch (launch -> activate) with graceful shutdown
• Telemetry: Real-time GPU monitoring and kernel performance metrics
• Backend Support: CPU simulation, CUDA via Numba/CuPy, and Metal via MLX (macOS)
• Performance Tiers: From uvloop (default) to Cython extensions

Installation

# Basic installation (CPU only)
pip install pydotcompute

# With CUDA support (NVIDIA GPUs)
pip install pydotcompute[cuda]

# With Metal support (macOS/Apple Silicon)
pip install pydotcompute[metal]

# With performance optimizations (uvloop - Linux/macOS)
pip install pydotcompute[fast]

# With Cython extensions (maximum performance)
pip install pydotcompute[cython]
python setup_cython.py build_ext --inplace

# Development installation
pip install -e ".[dev]"

Quick Start

import asyncio
from pydotcompute import RingKernelRuntime, ring_kernel, message

# Define message types
@message
class ComputeRequest:
    values: list[float]

@message
class ComputeResponse:
    result: float

# Define a ring kernel actor
@ring_kernel(
    kernel_id="compute",
    input_type=ComputeRequest,
    output_type=ComputeResponse,
)
async def compute_actor(ctx):
    while not ctx.should_terminate:
        msg = await ctx.receive()
        result = sum(msg.values)
        await ctx.send(ComputeResponse(result=result))

# Use the runtime (automatically uses uvloop for best performance)
async def main():
    async with RingKernelRuntime() as runtime:
        await runtime.launch("compute")
        await runtime.activate("compute")

        await runtime.send("compute", ComputeRequest(values=[1.0, 2.0, 3.0]))
        response = await runtime.receive("compute")

        print(f"Result: {response.result}")  # 6.0

asyncio.run(main())

Performance Tiers

PyDotCompute offers three performance tiers to match your use case:

Tier	Implementation	Latency (p50)	Use Case
1 (Default)	uvloop + FastMessageQueue	21μs	Async Python code
2	ThreadedRingKernel	~100μs	Blocking I/O, C extensions
3	CythonRingKernel	0.33μs queue ops	Multi-process IPC

Tier 1: Async (Default)

Automatically enabled when you import pydotcompute. Uses uvloop on Linux/macOS.

async with RingKernelRuntime() as runtime:
    # uvloop is auto-installed for 21μs latency
    await runtime.launch("my_kernel")
    await runtime.activate("my_kernel")

Tier 2: Threaded

For blocking operations or GIL-releasing C extensions:

from pydotcompute.ring_kernels import ThreadedRingKernel, ThreadedKernelContext

def blocking_kernel(ctx: ThreadedKernelContext):
    while not ctx.should_terminate:
        msg = ctx.receive(timeout=0.1)
        if msg:
            ctx.send(process(msg))

with ThreadedRingKernel("worker", blocking_kernel) as kernel:
    kernel.send(request)
    response = kernel.receive()

Tier 3: Cython (Maximum Performance)

For multi-process scenarios or Cython extensions:

from pydotcompute.ring_kernels import CythonRingKernel, is_cython_kernel_available

if is_cython_kernel_available():
    # 0.33μs queue operations
    with CythonRingKernel("fast_worker", my_kernel) as kernel:
        kernel.send(request)

Benchmarks

Message Latency

GPU Actors (1000 samples):
  p50:  63μs
  p95:  103μs
  p99:  131μs
  mean: 70μs

Graph Processing (PageRank)

Graph Size	CPU Sparse	GPU Batch	Speedup
1K nodes	6.8ms	64ms	CPU wins
5K nodes (dense)	256ms	200ms	GPU 1.28x
1M nodes	39.6s	4.25s	GPU 9.3x

Crossover: GPU wins at 50K+ nodes

Streaming Throughput

Scenario	GPU Actors	Advantage
Persistent state	Yes	No repeated GPU transfers
Transfer overhead	0%	vs 16-28% for batch
Best for	Long-running pipelines	Context preservation

Architecture

PyDotCompute Ring Kernel System
├── Ring Kernels          │ Performance Tiers      │ GPU Backends
│   • RingKernelRuntime   │ • uvloop (21μs)        │ CUDA:
│   • FastMessageQueue    │ • ThreadedRingKernel   │ • Numba JIT, CuPy arrays
│   • @ring_kernel        │ • CythonRingKernel     │ • Zero-copy DMA, PTX caching
│   • @message            │ • FastSPSCQueue        │ Metal (macOS):
│                         │                        │ • MLX, Unified memory
├─────────────────────────┴────────────────────────┴─────────────────
│ Memory: UnifiedBuffer (.host, .device, .metal), MemoryPool, Accelerator

Core Components

UnifiedBuffer

Transparent host-device memory management:

from pydotcompute import UnifiedBuffer
import numpy as np

buffer = UnifiedBuffer((1000,), dtype=np.float32)
buffer.allocate()

# Write on host
buffer.host[:] = np.random.randn(1000)
buffer.mark_host_dirty()

# Access on CUDA device (auto-syncs)
await buffer.ensure_on_device()
device_data = buffer.device

# Access on Metal/macOS (auto-syncs)
metal_data = buffer.metal  # MLX array

Ring Kernels

Persistent actors with message queues:

@ring_kernel(kernel_id="processor", queue_size=4096)
async def processor(ctx):
    while not ctx.should_terminate:
        msg = await ctx.receive(timeout=0.1)
        # Process message
        await ctx.send(response)

Lifecycle Management

async with RingKernelRuntime() as runtime:
    # Phase 1: Launch (allocate resources)
    await runtime.launch("my_kernel")

    # Phase 2: Activate (start processing)
    await runtime.activate("my_kernel")

    # Use the kernel...

    # Deactivate (pause) or Terminate (cleanup)
    await runtime.deactivate("my_kernel")
    await runtime.reactivate("my_kernel")

Metal Backend (macOS)

GPU acceleration on Apple Silicon using MLX:

from pydotcompute.backends.metal import MetalBackend, get_vector_add_kernel
import numpy as np

# Initialize backend
backend = MetalBackend()

if backend.is_available:
    # Copy data to Metal GPU
    a = backend.copy_to_device(np.array([1, 2, 3], dtype=np.float32))
    b = backend.copy_to_device(np.array([4, 5, 6], dtype=np.float32))

    # Use pre-built kernels
    add_kernel = get_vector_add_kernel()
    result = add_kernel(a, b)  # [5, 7, 9]

    # Copy back to host
    result_np = backend.copy_to_host(result)

    # Or compile custom kernels
    compiled = backend.compile_kernel(lambda x: x * 2 + 1)
    output = compiled(np.array([1, 2, 3], dtype=np.float32))

Project Structure

pydotcompute/
├── core/
│   ├── accelerator.py      # GPU device abstraction
│   ├── unified_buffer.py   # Host-device memory
│   ├── memory_pool.py      # Memory pooling
│   └── orchestrator.py     # Compute coordination
├── ring_kernels/
│   ├── runtime.py          # Main runtime (uvloop)
│   ├── message.py          # Message serialization
│   ├── queue.py            # Async message queues
│   ├── fast_queue.py       # O(1) priority queue
│   ├── lifecycle.py        # Kernel lifecycle
│   ├── telemetry.py        # Performance monitoring
│   ├── _loop.py            # uvloop auto-install
│   ├── sync_queue.py       # Threading queues
│   ├── threaded_kernel.py  # Tier 2 kernel
│   ├── cython_kernel.py    # Tier 3 kernel
│   └── _cython/            # Cython extensions
│       └── fast_spsc.pyx   # 0.33μs queue
├── backends/
│   ├── cpu.py              # CPU simulation
│   ├── cuda.py             # CUDA via Numba/CuPy
│   └── metal.py            # Metal via MLX (macOS)
├── compilation/
│   ├── compiler.py         # Kernel compilation
│   └── cache.py            # PTX caching
└── decorators/
    ├── kernel.py           # @kernel decorator
    ├── ring_kernel.py      # @ring_kernel decorator
    └── validators.py       # Runtime validation

Testing

# Run all tests (398 passing)
pytest

# Run with coverage
pytest --cov=pydotcompute

# Run only unit tests
pytest tests/unit/

# Skip CUDA tests (if no GPU)
pytest -m "not cuda"

# Skip Metal tests (if not on macOS)
pytest -m "not metal"

# Run benchmarks
python benchmarks/extended_benchmark.py
python benchmarks/pagerank_benchmark.py
python benchmarks/realtime_anomaly_benchmark.py
python benchmarks/metal_benchmark.py  # macOS only

Requirements

• Python >= 3.11
• numpy >= 1.26.0
• msgpack >= 1.0.0

Optional Dependencies

Package	Purpose
uvloop	20-40% faster event loop (Linux/macOS)
cupy-cuda12x	CUDA array operations
numba	GPU kernel JIT compilation
pynvml	GPU monitoring
mlx	Metal GPU acceleration (macOS only)
cython	Maximum performance queues

Disabling uvloop

If you need to disable uvloop auto-installation:

PYDOTCOMPUTE_NO_UVLOOP=1 python my_script.py

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines and docs/IMPLEMENTATION_PLAN.md for the project roadmap.

License

Apache License 2.0 - see LICENSE file for details.

Related

• DotCompute - Original .NET implementation
• Numba CUDA - Python CUDA JIT
• CuPy - NumPy-compatible GPU arrays
• MLX - Apple's ML framework for Apple Silicon
• uvloop - Fast asyncio event loop