reactor-realtime-engine 0.1.1

Standalone realtime inference engine: the RealtimeInterface contract, a default scheduling loop, and a CPU reference engine — driven entirely through inbox/outbox queues

realtime_engine — a standalone realtime inference engine

A standalone realtime inference engine, driven entirely through inbox/outbox queues — so any caller matching the contract can run it.

This follows the design's build order: prove the engine on its own first (the queue boundary gives isolation now, and a path to a process/IPC split later without a process boundary today). Full design write-up lives in Notion.

Layout

Standard src/ layout: src/realtime_engine/ (package) + tests/ + pyproject.toml.

module	what
contract.py	RealtimeInterface (Protocol) + EngineManifest / SessionInit / StepResult / AdmissionError — Level 1: manifest/attach/step/finalize/detach
queues.py	the boundary: inbox msgs (Attach/Detach/Reset/Action) + Chunk; Inbox/Outbox (asyncio.Queue wrappers)
serve.py	default_serve — the Level-2 loop a Level-1 engine reuses (drain → batched step → outbox → finalize → tick; generate-from-default); run_engine picks an engine's own serve if it has one
cpu_reference.py	CpuReferenceEngine — numpy-only deterministic fake (no GPU/ML); proves admission, batching, invariance, distinctness
tests/test_engine.py	standalone tests that feed the inbox and drain the outbox

Develop & test

cd backend/realtime_engine
pip install -e ".[dev]"
ruff check src tests && black --check src tests && isort --check-only src tests && mypy src/realtime_engine
pytest -q

Tests cover: distinctness (co-resident sessions differ), invariance (a session is identical solo vs batched), admission (over-capacity attach raises AdmissionError), finalize, and output shape — all with the engine running standalone. Lint/type/test also run in CI (.buildkite/pipeline.yml).

Contract recap

• Level 1 (every engine): manifest / attach / step(due) -> [StepResult] / finalize / detach. step is batched (one StepResult per due session).
• Level 2 (the loop): optional serve(inbox, outbox). Omit it → default_serve. Implement it → own loop (e.g. an sglang-style scheduler).
• Conditioning arriving via Action is action-class (already client→action-translated by the caller); the engine stays ignorant of client semantics.

Next (not built yet)

• A real GPU engine impl behind the same contract (Waypoint stages on a ComposedEngine).
• The runtime caller: a ReactorModel that pumps the inbox (state → Action) and routes the outbox into streaming.
• Open validation item (flagged in review): measure whether the outbox queue adds latency vs a direct callback.