holoscript-holoserve

holoscript-holoserve is the public Python distribution for HoloServe, the native sovereign PyTorch serving runtime for HoloRunner checkpoints. It exposes OpenAI-compatible completion, chat, and SSE routes while retaining the machine-checkable serving identity:

{
  "backend": "pytorch-holo",
  "sovereign": true,
  "llama_cpp": false,
  "gguf": false
}

This is a v0-preview for external builders, local operators, and agent frameworks evaluating HoloRunner models. Its API can change during the 0.x line.

Install

python -m pip install holoscript-holoserve
holoserve --help

The distribution includes the resident HTTP server, checkpoint-compatible HoloRunner decoder, byte-BPE tokenizer, constrained grammar engine, epistemic router, optional epistemic sidecar runtime, and proof-bearing consumer payload broker. It does not ship a checkpoint, tokenizer data, CUDA runtime, credentials, or machine-specific service files.

Run

Callers bring their own checkpoint, tokenizer bins, and content-addressed custody receipt:

holoserve \
  --model-name holorunner-s0 \
  --ckpt /srv/holoserve/holorunner-s0/ckpt.pt \
  --bins /srv/holoserve/holorunner-s0/bins \
  --snapshot-dir /srv/holoserve/private-snapshots \
  --custody-receipt /srv/holoserve/holorunner-s0/custody-receipt.json \
  --expected-custody-sha256 sha256:REPLACE_WITH_PINNED_RECEIPT_DIGEST \
  --host 127.0.0.1 \
  --port 8099

Replace the digest placeholder with the independently computed receipt digest; the CLI fails closed on malformed or unbound custody. Use --model-manifest for a caller-owned multi-model manifest. HoloServe validates the pinned checkpoint and tokenizer artifacts before admitting a model, then exposes /health, /v1/models, /v1/completions, /v1/chat/completions, /props, and /slots.

curl http://127.0.0.1:8099/health

The health report and response holo metadata are the agent-operable receipt surface. A resolver must reject the endpoint unless the typed sovereignty fields and artifact registry agree.

Starting with 0.1.1, /health.execution_hardware binds the running package, node, architecture, resident model parameter device, and CUDA device identity. Every successful completion also includes holo.execution (holoscript.holoserve-execution-receipt.v1) whose server_request_id exactly matches the OpenAI-compatible response id. gpu_execution is true only for model generation when the resident parameters and CUDA target agree and process-local CUDA telemetry can be read. Responses served by the deterministic executor explicitly report execution_kind: "deterministic-executor" and never claim GPU model work.

Starting with 0.1.2, callers may opt into the consumer payload broker on an individual request:

{
  "model": "holorunner-s0",
  "prompt": "Return 125 multiplied by 8 as a bare integer.",
  "holo_pipeline": "broker-v1"
}

The broker is default-off. It emits a payload only when a structural executor or content-addressed closed-world fact proves ownership. Tool misses require the separately admitted epistemic sidecar before decoder fallback, and every non-decoder payload carries an executor receipt in response holo metadata.

Validation

From a source checkout:

python -m pytest packages/holoserve-py/tests
python scripts/check_holoserve_cold_consumer.py
node scripts/check-package-public-consumption.mjs \
  --py-package packages/holoserve-py

The cold-consumer gate builds a wheel, installs that wheel in a fresh virtual environment, proves imports resolve from installed package files rather than the source checkout, exercises tokenizer/model/grammar code, and verifies the CLI help path.

Support and rollback boundary

Supported in 0.1.x: HoloRunner S0-family checkpoints matching the packaged decoder architecture, byte-BPE tokenizer metadata, CPU/CUDA devices supported by the caller's PyTorch installation, blocking completions, chat flattening, and SSE streaming.

Known limitations: the package does not train models, provision GPUs, install drivers, download checkpoints, manage a system service, or grant artifact custody. Operators own those inputs and recovery. Multi-process batching and remote checkpoint acquisition are unsupported.

Rollback after a future upgrade by pinning the previous known-good 0.x wheel and restarting the operator-owned service:

python -m pip install --force-reinstall holoscript-holoserve==0.1.0

The package boundary is public and portable; founder-local storage, private workspace state, credentials, deployment adapters, and hardware policy remain outside the distribution.