pydecide

decide is one Python client for typed decisions - Choice, Score and Noul - over any "System One" decision model: TypeSafe's hosted Jev, OpenRouter's Decisions endpoint, the open-weight laya family (PyTorch and MLX), any sentence-transformers CrossEncoder, and a JSON-prompted LLM fallback. A Client takes an ordered list of backends and a confidence policy: if a backend errors or answers with low confidence, the next backend is tried. The library also ships a small HTTP server that speaks TypeSafe's wire protocol, so existing TypeSafe clients can point at a local model instead.

Install

uv tool install pydecide   # the decide CLI on your PATH, from any directory
uv add pydecide            # inside a project

With pip: pip install pydecide / pipx install pydecide. Either way, this includes the CLI, the hosted backends and the server.

Local model backends need an extra:

uv tool install "pydecide[all]"

With pip: pip install "pydecide[all]". Quote the brackets in zsh, since it otherwise tries to glob them. This pulls in PyTorch and MLX, about 2 GB.

The mlx extra depends on laya-mlx, which requires Python 3.11 or newer; on 3.10 the extra installs nothing and the laya_mlx backend is unavailable.

Quickstart: zero configuration

Install a local backend and decide ask works right away, with no environment variables at all:

uv tool install "pydecide[mlx]"    # Apple Silicon
# or
uv tool install "pydecide[laya]"   # any platform (PyTorch)

decide ask "I had a rough day, everything broke" --noul "Is the writer doing well?" --choice good,bad

NAME    TYPE    ANSWER  PROBABILITIES
choice  choice  bad     good=0.01 bad=0.99
noul    noul    false   0.01
backend=laya_mlx route=laya_mlx:ok latency=33.1ms

This is the real output of that command, run with laya_mlx installed and none of DECIDE_LOCAL_MODEL, TYPESAFE_API_KEY, OPENROUTER_API_KEY, OPENAI_API_KEY, DECIDE_LLM_BASE_URL or DECIDE_BACKENDS set.

The Python client works the same way. Client.from_env() builds a backend chain from whatever is installed and configured in the process environment; with a local backend installed, it needs no variables either. Pass an explicit env= mapping instead (e.g. in tests) to configure from something other than os.environ:

from decide import Client, Choice, Score, Noul

client = Client.from_env()

r = client.decide(
    state={"ticket": "I was charged twice, please refund."},
    questions={
        "team": Choice(
            "Which team handles this?",
            {
                "billing": "charges, refunds",
                "engineering": "bugs and outages",
                "sales": "pricing and upgrades",
            },
        ),
        "severity": Score("How severe is this?", ["minor", "degraded", "blocked"]),
        "refund": Noul("Does the customer ask for money back?"),
    },
)

r.choices["team"].choice  # "billing"
r.choices["team"].probabilities  # {"billing": 1.0, "engineering": 0.0, "sales": 0.0}
r.scores["severity"].score  # 1.5522  (expected level index, 0..len(levels)-1)
r.scores["severity"].probabilities  # [0.0197, 0.4083, 0.5719]
r.nouls["refund"].noul  # 0.9461
r.meta.backend  # "laya_mlx"
r.meta.latency_ms  # 37.1
r.meta.route  # ["laya_mlx:ok"]

This was run against the laya_mlx backend with no environment variables set at all, using its own default checkpoint; every value above is the real output of that run, not illustrative.

from_env picks the chain in this order, using whatever is installed and/or configured: laya_mlx or laya, whichever is importable, with its own default model unless DECIDE_LOCAL_MODEL overrides it; typesafe (if TYPESAFE_API_KEY is set); openrouter (if OPENROUTER_API_KEY is set); llm (if DECIDE_LLM_BASE_URL or OPENAI_API_KEY is set). A local backend, when installed, is tried first, with the hosted backends as fallback. Set DECIDE_BACKENDS="laya,typesafe" to override the order explicitly. If nothing is importable or configured, from_env raises ConfigError telling you to install a local backend or set a hosted one's API key.

AsyncClient has the same surface, awaited: await client.decide(...), await client.decide_batch(...), AsyncClient.from_env(...).

Backends

crossencoder is built by hand, for example:

from decide import Client, Choice
from decide.backends.crossencoder import CrossEncoderBackend

backend = CrossEncoderBackend("cross-encoder/ms-marco-MiniLM-L6-v2")
client = Client([backend])

r = client.decide(
    "I was charged twice for the same order last week, please refund the duplicate charge.",
    {
        "team": Choice(
            "Which team should handle this?",
            {
                "billing": "Charges, invoices, payment problems, refunds",
                "eng": "Bugs, crashes, broken features",
                "shipping": "Delivery status, delays, lost packages",
            },
        )
    },
)
r.choices["team"].choice  # "billing"
r.choices["team"].probabilities  # {"billing": 0.947, "eng": 0.019, "shipping": 0.034}

Environment variables: overrides and hosted-backend keys

None of these are required to get started - see Quickstart: zero configuration above. They either override a local backend that from_env already auto-selects once it's installed, or supply the API key a hosted backend needs to be auto-selected at all.

Gating and fallback

from decide import Gate

Gate(
    min_confidence=0.0,  # top probability of a Choice, max(noul, 1-noul) for a Noul
    per_question=None,  # optional {"question_name": threshold} overrides
    on_error="next",  # or "raise" to stop the chain on the first BackendError
)

For each backend in order: call it. On BackendError with on_error="next", append "<name>:error" to meta.route and try the next backend (with on_error="raise", the error propagates immediately instead). If the response fails the gate, append "<name>:low_confidence:<question>=<value><threshold>" and try the next backend. If it passes, append "<name>:ok" and return. Score answers are never gated (there is no single confidence number for an expected value over levels).

If every backend is exhausted without a passing response, the best response seen so far (highest minimum confidence across its gated answers) is returned, with meta.route[-1] == "<name>:accepted_low_confidence". If no backend produced any response at all, Client.decide raises AllBackendsFailed(route, errors).

Route strings you will see in meta.route:

• "<name>:ok" - the backend answered and passed the gate.
• "<name>:error" - the backend raised a BackendError.
• "<name>:low_confidence:<question>=<value><threshold>" - the backend answered but at least one gated question fell below its threshold.
• "<name>:accepted_low_confidence" - appended once, at the end of the route, when no backend passed the gate and the best low-confidence response was returned instead.

Client.decide_batch/AsyncClient.decide_batch run the same chain per input state, preserving input order; a state that clears the gate on an earlier backend is not sent to later ones. Client.decide_batch uses a backend's real batch path when capabilities().batch is true, looping decide per request otherwise. AsyncClient.decide_batch always loops adecide per state, concurrently via asyncio.gather; it does not use a backend's batch path in v1. If any state in the batch is left unrouted, AllBackendsFailed carries partial (every Response that did resolve, keyed by input index) and failed (the route so far for every state that did not), so the resolved siblings are not silently lost.

Server: point TypeSafe's SDK at a local model

DECIDE_LOCAL_MODEL=aac6fef/laya-multilingual-mlx decide serve --backends laya_mlx --port 8811

GET /health reports liveness and the configured backend names; GET /v1/models lists them in an OpenAI-style shape; POST /v1/systemone takes a TypeSafe SystemOneRequest body and returns a SystemOneResponse-shaped body plus a decide extension carrying our own backend/route metadata. Errors come back as {"error": {"message": ..., "type": ...}} with a matching HTTP status.

If --api-key/DECIDE_API_KEY is set, requests must send a matching Authorization: Bearer <token> header; the key must be ASCII, since HTTP header bytes are latin-1 decoded by the server before comparison.

A raw request against the running server above:

curl -s -X POST http://127.0.0.1:8811/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"ticket": "I was charged twice, please refund."},
    "questions": {
      "team": {"type": "choice", "instructions": "Which team handles this?",
               "criteria": {"billing": "charges, refunds", "engineering": "bugs and outages", "sales": "pricing and upgrades"}},
      "severity": {"type": "score", "instructions": "How severe is this?", "criteria": ["minor", "degraded", "blocked"]},
      "refund": {"type": "noul", "instructions": "Does the customer ask for money back?"}
    }
  }'

{
  "model": "aac6fef/laya-multilingual-mlx",
  "answers": {
    "team": {"type": "choice", "choice": "billing", "confidence": 1.0,
              "probabilities": {"billing": 1.0, "engineering": 0.0, "sales": 0.0}},
    "severity": {"type": "score", "score": 1.5522, "confidence": 0.5719,
                 "legend": {"0": "minor", "1": "degraded", "2": "blocked"},
                 "probabilities": {"0": 0.0197, "1": 0.4083, "2": 0.5719}},
    "refund": {"type": "noul", "noul": 0.9461}
  },
  "usage": {"input_tokens": 0, "output_tokens": 0},
  "decide": {"backend": "laya_mlx", "latency_ms": 50.5, "route": ["laya_mlx:ok"]}
}

And the same request through TypeSafe's own SDK, pointed at the local server (no API key is needed since this server has none configured, but the SDK requires a non-empty string):

from typesafe_sdk import TypeSafeClient, Choice, Score, Noul

client = TypeSafeClient(api_key="anything", base_url="http://127.0.0.1:8811")

resp = client.system_one(
    state={"ticket": "I was charged twice, please refund."},
    questions={
        "team": Choice(
            instructions="Which team handles this?",
            criteria={
                "billing": "charges, refunds",
                "engineering": "bugs and outages",
                "sales": "pricing and upgrades",
            },
        ),
        "severity": Score(
            instructions="How severe is this?", criteria=["minor", "degraded", "blocked"]
        ),
        "refund": Noul(instructions="Does the customer ask for money back?"),
    },
)

model='aac6fef/laya-multilingual-mlx' usage=Usage(input_tokens=0, output_tokens=0)
answers={'team': ChoiceAnswer(type='choice', choice='billing', confidence=1.0,
  probabilities={'billing': 1.0, 'engineering': 0.0, 'sales': 0.0}),
 'severity': ScoreAnswer(type='score', score=1.5522, confidence=0.5719,
  legend={0: 'minor', 1: 'degraded', 2: 'blocked'},
  probabilities={0: 0.0197, 1: 0.4083, 2: 0.5719}),
 'refund': NoulAnswer(type='noul', noul=0.9461)}

TypeSafe's SDK parsed our server's response without modification: it is a genuine SystemOneResponse, not a hand-shaped dict.

CLI

--choice, --score and --noul question names are optional. An unnamed flag is auto-named after the flag itself (choice, score, noul); a second unnamed flag of the same type becomes choice2, score2, noul2, and so on:

decide ask "I had a rough day" --noul "Is the writer doing well?" --choice good,bad

NAME    TYPE    ANSWER  PROBABILITIES
choice  choice  bad     good=0.07 bad=0.93
noul    noul    false   0.02
backend=laya_mlx route=laya_mlx:ok latency=65.3ms

Name a question explicitly with NAME=... to control what shows up in the table and in --json; named and unnamed questions can be mixed on the same command line:

decide ask "I was charged twice, please refund." \
  --choice "team=billing,engineering,sales" \
  --score "severity=minor,degraded,blocked" \
  --noul "refund=Does the customer ask for money back?"

NAME      TYPE    ANSWER    PROBABILITIES
team      choice  billing   billing=0.93 engineering=0.02 sales=0.05
severity  score   degraded  1.09 (degraded)
refund    noul    true      0.87
backend=laya_mlx route=laya_mlx:ok latency=34.8ms

--json prints a machine-readable payload instead of the table. --min-confidence and --model are also available on ask.

decide backends

NAME          INSTALLED  CONFIGURED  INSTALL
typesafe      yes        no
openrouter    yes        no
laya          yes        default
laya_mlx      yes        default
crossencoder  yes        n/a
llm           yes        no

Every backend with INSTALLED no gets an INSTALL column naming the exact command to add it, e.g. pip install "pydecide[laya]". For laya and laya_mlx, CONFIGURED reads default when installed with no DECIDE_LOCAL_MODEL override (it will be auto-selected with its own default model), yes when DECIDE_LOCAL_MODEL is set, and no only when the backend isn't installed at all.

decide serve --backends a,b --host 127.0.0.1 --port 8811 [--api-key TOKEN] [--min-confidence FLOAT] runs the HTTP server described above.

What the probabilities mean

Every probability in a ChoiceAnswer, ScoreAnswer or NoulAnswer is whatever the backend reported; decide does not calibrate, smooth or verify it. What that means differs by backend: typesafe, openrouter and laya/laya_mlx are purpose-built decision models, but their outputs are still self-reported by the model and not audited by this library. The crossencoder backend turns a relevance reranker's raw logits into a softmax or sigmoid - the result is a normalized score forced to distribute mass over the supplied candidates, not a calibrated probability; a Choice will still pick a winner even when every candidate is a bad fit, and a Noul of 0.9 does not mean the condition holds 90% of the time. The llm backend is the least trustworthy of all: it prompts a general chat model to estimate its own confidence in JSON, with no guarantee the model attends to every candidate or keeps its numbers well calibrated. Treat all of this accordingly - as a signal to gate and fall back on, not as ground truth.

Status

pydecide is at 0.1.1. The public API may still change before a 1.0 release.

License

MIT, see LICENSE.