laya-hermes

A Hermes Agent plugin that gives the agent access to Laya — a local, non-generative "System 1" typed-decision model. Laya answers choice / score / boolean (noul) questions with calibrated probabilities in a single forward pass (~5–15 ms on Apple Silicon), fully offline. Use it for routing, triage, gating, and moderation decisions instead of spending LLM tokens.

What the plugin provides

• laya_decide tool — run typed questions against a state (text, JSON, or conversation list). Custom questions or built-in presets (router, guard, moderation, triage). Multiple questions are batched in one forward pass.
• laya_status tool — backend detection, installed packages, live settings, loaded models, metrics.
• /laya slash command — ad-hoc decisions plus status, stats, setup, config, set (try /laya help).
• laya:laya-decisions skill — teaches the agent when to prefer Laya over in-LLM reasoning.
• Opt-in pre_llm_call routing hint — with routing_hint on, Laya rates each user message's complexity and injects a short hint when it's confident the request is simple. Hint only; it never blocks or overrides your model.
• Opt-in output filtering — with filter_output on, Laya screens successful oversized tool/terminal outputs and truncates ones it is confident are disposable (install spam, progress bars) to head+tail with a marker. Failures and ambiguous output always pass through untouched. Laya doesn't generate text, so this truncates — it never summarizes.
• Context engine (smart compaction) — opt in with hermes config set context.engine laya then /reset. During context compression, Laya judges each stale tool call/result pair (keep / truncate / drop) instead of Hermes pruning by age alone — so a test failure from three turns ago survives while install spam is dropped. Fully local: unlike hermes-jev-compact (which pioneered this design with the hosted Jev API), no transcript leaves the machine and each verdict is free. Safety: the proactive hot path stays deterministic, and any error, invalid transcript, or under-min_reduction_ratio pass falls back to the built-in prune — worst case is stock Hermes behavior. Caveat: Laya's ~1024-token window means per-unit judgments (goal + recent tail + the call), not whole-transcript reasoning.
• Session metrics — /laya stats shows decisions, avg latency, truncations, and estimated tokens saved. In-memory; resets when Hermes restarts.

Backends

auto (default) prefers mlx → coreml → torch on Apple Silicon, torch elsewhere.

Install

From PyPI (once published):

~/.hermes/hermes-agent/venv/bin/pip install hermes-laya
hermes plugins enable laya

Or from GitHub (note the #hermes_laya subdir — the plugin lives in the package folder):

hermes plugins install pavlealeksic/laya-hermes#hermes_laya --enable

That's it. The right backend package for your platform (laya-mlx on Apple Silicon) self-installs into Hermes' Python environment on first use — no manual pip step. The model checkpoint (~650 MB) then downloads from Hugging Face on the first decision. Opt out with /laya set auto_install false and install manually, or run /laya setup to trigger the install on demand.

For local development, symlink hermes_laya/ into ~/.hermes/plugins/laya/ and hermes plugins enable laya.

Configuration

All settings are adjustable live from inside Hermes — no restart needed:

/laya config                          # show every setting, its value, and its source
/laya set filter_output true          # enable output filtering immediately
/laya set routing_hint true           # enable pre-LLM-call complexity hints
/laya set model typed-decisions       # switch checkpoint (applies to next decision)

Settings persist in Hermes' config.yaml under plugins.entries.laya.settings and are declared in the plugin's config_schema, so Hermes' settings UI can render them too. Environment variables still work and override settings: precedence is env var (LAYA_*) → Hermes setting → default.

Deliberately not included

• Per-turn main-model routing — Hermes v0.21 has no plugin seam for switching the main loop's model (llm.model_override covers only a plugin's own ctx.llm calls). The routing hint is the honest approximation until Hermes adds one.
• Skill routing — Hermes already progressive-discloses skills (compact index, load on demand); there is no skill-context bloat to fix.
• Output summarization — Laya is non-generative; filtering truncates, it can't rewrite.

Example

Ask the agent something like "use laya to triage this ticket: …", or call the tool shape directly:

{
  "state": "I was billed twice this month and support never replied.",
  "questions": {
    "department": {"type": "choice", "instructions": "Which team handles this?",
                   "criteria": {"billing": "charges and refunds", "technical": "bugs and outages"}},
    "urgency": {"type": "score", "instructions": "How urgent?",
                "criteria": ["low", "medium", "high", "critical"]},
    "refund_requested": {"type": "noul", "instructions": "Does the customer ask for money back?"}
  }
}

→ answers.department.choice = "billing", answers.refund_requested.noul ≈ 0.9, plus per-option probabilities, confidence, and action.act_probability.

Development

python3 -m unittest discover -s tests -v     # unit tests (stubbed backend, no download)
python3 -m venv .venv && .venv/bin/pip install laya-mlx
.venv/bin/python tests/smoke.py              # end-to-end against the real model
python -m build                              # build sdist + wheel into dist/

Publishing a release (maintainer)

The repo ships .github/workflows/publish.yml using PyPI Trusted Publishing (no stored tokens). One-time setup, then releases are automatic:

1. On pypi.org: create an account → Account settings → Publishing → Add a new pending publisher → fill in: PyPI project name hermes-laya, owner pavlealeksic, repository laya-hermes, workflow publish.yml, environment pypi. (A "pending publisher" creates the project on first publish — no need to pre-create it.)
2. On GitHub: repo Settings → Environments → New environment named pypi (optionally add required reviewers for a manual gate).
3. Cut a release: bump version in both pyproject.toml and hermes_laya/plugin.yaml, commit, then gh release create v1.2.0 --generate-notes — the workflow runs tests, builds, publishes.
4. Verify: pip install hermes-laya==1.2.0 in a scratch venv.

Manual fallback if you prefer: python -m build && twine upload dist/* with a PyPI API token (pip install twine).

Credits & license

Plugin code: Apache-2.0. Laya model and runtimes by Convai Innovations (Apache-2.0); MLX/Core ML ports by mizorewww. Not affiliated with Nous Research, Convai Innovations, or Apple.