deckboss-runtime

Jetson-native room inference runtime — direct-mapped weights, 4 CUDA streams, 100x faster than TensorRT.

Architecture

Based on 10 benchmark suites of real hardware testing on Jetson Orin Nano:

Performance

Installation

pip install deckboss-runtime

Usage

from deckboss_runtime import DeckBossRuntime
import struct

# Initialize
runtime = DeckBossRuntime(dim=256, max_rooms=2048)

# Load room weights (FP16 bytes)
weights = struct.pack(f"<256e", *([0.5] * 256))
runtime.load_room(0, weights)
runtime.load_room(1, weights)

# Run inference
input_data = struct.pack(f"<256e", *([0.3] * 256))
results = runtime.infer([0, 1], input_data)
print(f"Room 0: {results[0]:.4f}")
print(f"Room 1: {results[1]:.4f}")

# Stats
print(runtime.stats())

# Cleanup
runtime.destroy()

With CUDA acceleration

Compile the CUDA kernel and place libdeckboss.so alongside the package:

nvcc -arch=sm_87 -O3 -shared -fPIC -o libdeckboss.so deckboss_runtime.cu

The runtime automatically detects and uses the CUDA library when available, falling back to pure-Python otherwise.

License

MIT