decisionsmith

Use and fine-tune System One models (Jev, Laya) on your data. Start with an LLM. End with a fast decision model you trained. One line in between.

uv add "decisionsmith[all]"

import decisionsmith as ds

h = ds.harness(Ticket, teacher="claude-sonnet-5", student="laya")
h("You charged me twice, refund now!")
# Ticket(team='billing', wants_refund=True)

That's it. Ticket is a normal Pydantic model:

from typing import Annotated, Literal

from pydantic import BaseModel, Field

import decisionsmith as ds


class Ticket(BaseModel):
    team: Annotated[
        Literal["billing", "technical", "sales"],
        ds.Options(billing="payments, refunds", technical="bugs, outages", sales="pricing"),
    ]
    wants_refund: bool = Field(description="Does the customer ask for their money back?")

Literal / Enum → pick one · bool → yes/no · Annotated[Literal["low", "medium", "high"], ds.Scale] → a rating.

Why

Decision models like Laya are fast and free to run, and weak zero-shot (0.36 accuracy on typed-decisions before fine-tuning, 0.77 after, per Laya's benchmarks). LLMs are good zero-shot, and slow and paid per call. decisionsmith starts with the LLM, logs every answer, fine-tunes the fast model on those answers, and moves traffic to it field by field once the numbers say it's ready.

So don't trust a System One model zero-shot on your task. Let a normal LLM make the decisions first (mode="teacher" or "shadow"), behind the guardrails decisionsmith already has: answers validated against your schema, the text fenced as data, a fallback when an engine fails, and a per-field confidence threshold plus a 5% audit before any field moves to the student.

The loop

texts = open("texts.txt", encoding="utf-8").read().splitlines()  # your texts, one per line
h = ds.harness(Ticket, teacher="claude-sonnet-5", student="laya", mode="shadow")  # teacher answers, student measured
h.many(texts)  # every decision is logged (decisions.db)
print(h.status())  # per field: agreement, sure rate, accuracy when sure, what to do next
h.adapt()  # calibrate the student's confidence + pick safe thresholds (seconds, any engine)
h.finetune()  # train Laya on the logged answers; switches only if it beats the current student

team:         shadow · student agrees 94% · sure on 71% · accuracy when sure 97% (vs teacher) -> ready for cascade
wants_refund: shadow · student agrees 81% · 420 labelled -> ready to finetune: run h.finetune()

Then move a field when it's ready: mode={"team": "cascade", "wants_refund": "shadow"}.

Four modes, per field

If an engine fails, the other one answers and the result says sure=False. Nothing crashes in the middle.

Connect anything

Any engine can be the teacher or the student.

Fine-tune with labels you already have

decisionsmith finetune tickets.csv --schema app.py:Ticket --out runs/v1

A CSV needs a text column and one column per field. Small data (under 1,000 rows) trains only the decision head, fast even on a laptop; bigger data trains the whole model (use a GPU, or the Kaggle/Colab notebook in examples/). You get runs/v1/: a plain Laya checkpoint (laya.load("runs/v1") works), a report comparing it with the base model on held-out data, and a go/no-go verdict with the reasons.

Or from Python, and keep the result as a versioned folder you can load anywhere:

model = ds.model(["billing", "technical", "sales"])
model.train("tickets.csv")
print(model.evaluate("test.csv"))  # held-out numbers and go/no-go
m = ds.load(model.save("models/ticket"))  # models/ticket-v1

In production, ds.harness(m, teacher=llm, collect=0.1) keeps a sample of real texts, and ds.golden("decisions.db", teacher=llm, schema=labels) turns them into the next labelled dataset (golden, collect).

Compare engines on your data

decisionsmith bench tickets.csv --schema app.py:Ticket --engines claude-sonnet-5,jev,laya,laya:./runs/v1

From agents

uvx --from "decisionsmith[all]" decisionsmith mcp gives Claude Code, Codex, Cursor and other MCP clients the harness, bench, status, finetune and label as tools. There's also a Claude Code plugin; see docs/agents.md.

Docs

Quickstart · Fine-tuning · Guide: engines, modes, status, adapt, bench · FAQ · Integrations

Runnable examples (gallery, every one runs offline) and notebooks.

Built on Laya (Apache-2.0) by Nandakishor M / Convai Innovations. Jev and TypeSafe are trademarks of their owners; decisionsmith is an independent project, not affiliated with TypeSafe AI or Convai Innovations.