laya-serve

Jev-compatible HTTP server for Laya System One decision models. Point any Jev client at this server and get typed choice / score / noul answers from local Laya weights instead of the TypeSafe API.

Migration

pip install "laya[serve]"
LAYA_PRELOAD=1 laya-serve  # binds 0.0.0.0:8000, preloads all checkpoints

Quickstart

pip install "laya-serve[inference]"
LAYA_SERVE_BACKEND=laya LAYA_SERVE_PRELOAD=true laya-serve

curl -X POST localhost:8000/v1/systemone \
  -H "Content-Type: application/json" -d '{
    "state": "Help! My payouts have been failing for 3 days.",
    "model": "jev-latest",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payments, invoicing, refunds",
          "technical": "Bugs, outages, integrations",
          "sales": "Pricing, upgrades, new accounts"
        }
      },
      "is_urgent": {"type": "noul", "instructions": "The message conveys urgency?"}
    }
  }'

Without weights (API development, CI):

pip install -e ".[test]"
LAYA_SERVE_BACKEND=fake laya-serve  # deterministic uniform answers

Endpoints

Errors use {"error": {"message", "field"}} with Jev status codes (401 bad key, 422 validation, 429 rate limit with a Retry-After header when LAYA_SERVE_RATE_LIMIT_PER_MINUTE > 0, 529 transient overload with a Retry-After header, 500 unexpected backend failure with internals logged server-side).

Configuration (LAYA_SERVE_ env prefix)

Accepted model names out of the box: jev-latest, jev-preview, jev-1.13.0, jev-1.13, laya, laya-latest, plus the serving model id itself. Anything else is a 422 (fail fast on typos, like Jev).

Jev compatibility notes

Verified against docs.typesafe.ai and laya 0.3.x source. Deliberate, documented divergences:

1. Laya-only fields are stripped: action on every answer, and confidence on noul answers. Responses contain exactly the Jev fields.
2. model echoes the serving checkpoint (e.g. laya-english), the same way Jev echoes the resolved version id (jev-1.13.0).
3. output_tokens is 0. Laya never generates tokens; this layer does not invent counts. input_tokens is the backend's token count.
4. Confidence formula differs: Laya uses entropy-based 1 - H(p)/log(k); Jev documents a peak-based normalization. Same probabilities can yield different confidence values — calibrate thresholds against Laya, not Jev.
5. Validation: choice ≤ 255 options, score 2–10 levels, model required, non-empty questions — all 422. Context budgets mirror Jev's 64k (state + all questions) / 32k (state + longest question) accounting and surface as 422, as do option sets overflowing the per-question head budget; request sizes use a deterministic word count on every backend so the 200/422 boundary is stable. The checkpoint still truncates per-question sequences at its own max_len.
6. Score legend/probabilities keys are strings on the wire ({"0": …}), matching Jev HTTP.
7. choice criteria as a list is accepted leniently (mapped to {label: None}); Jev requires a map.

Development

python -m venv .venv && .venv/bin/pip install -e ".[test]"
.venv/bin/python -m pytest -q

Or with uv (reproducible, via uv.lock):

uv sync --extra test
uv run pytest -q

Lint/format via ruff and hooks via pre-commit:

uvx ruff check src tests && uvx ruff format --check src tests
pre-commit install && pre-commit run --all-files

Documentation

Versioned docs (Zensical + mike): https://stiermid.github.io/laya-serve/ (latest tracks the newest v* tag with the v stripped, dev tracks master — see docs/versioning.md).

uv sync --group docs
uv run zensical serve        # local preview
uv run zensical build --strict  # same check CI runs

License

Apache-2.0 — see LICENSE.