A decision model takes a state (text or JSON) and one or more named questions, and answers each with a probability for every option in a single forward pass. No text is generated, so there is no sampling and no temperature. Three question types cover most decisions:

Endor serves these models and lets you fine-tune them on your own labeled data, with a training loop you drive from Python while Endor runs the GPUs.

Install

pip install endor
export ENDOR_API_KEY=edk_...

Python 3.10 and up. Dependencies: httpx and pydantic.

The SDK calls production (https://api.endor.thetokencompany.com). For another environment, such as staging, set ENDOR_BASE_URL=https://staging.api.endor.thetokencompany.com or pass EndorClient(base_url=...).

API keys belong to an organization (edk_..., created in the dashboard); everything you create belongs to that org.

Decisions

Every decision goes through a project, which records it. Create one with a base model and it answers at once:

import endor
from endor import Choice, Noul, Score

client = endor.EndorClient()
client.projects.create("tickets", base_model="decider-2b")   # once

res = client.system_one(
    {"subject": "Charged twice", "body": "I was billed two times for my March invoice."},
    {
        "department": Choice(instructions="Which team should handle `body`?",
                             criteria={"billing": "Payments, refunds", "technical": "Bugs, outages", "sales": None}),
        "urgent": Noul(instructions="Does the customer need an answer today?"),
        "frustration": Score(instructions="How frustrated is the customer?", criteria=["Calm", "Annoyed", "Angry"]),
    },
    model="tickets/v1",          # a saved model; "tickets/base" for the base model
)

res.choices["department"].choice          # "billing"
res.choices["department"].probabilities   # {"billing": 0.91, "technical": 0.07, "sales": 0.02}
res.nouls["urgent"].noul                  # 0.83
res.scores["frustration"].score           # 1.4 (expected level), plus .probabilities and .legend
res.model, res.usage.input_tokens         # the model that answered, e.g. "tickets/v1"

A bare base model id such as "decider-2b" raises ModelRequiresProjectError (422 model_requires_project). There is no default model: pass model= or set one with EndorClient(model="tickets") or ENDOR_DEFAULT_MODEL, else system_one raises EndorError before sending anything.

Questions can also be plain dicts in the same shape. Every question is answered independently against the same state. Refer to parts of the state with backticked paths, as in "Is `body` angry?".

Decisions are TypeSafe's API (POST /v1/systemone). The SDK has its own Noul, Choice, Score, answer and SystemOneResponse types, which send and read exactly TypeSafe's wire format, so it doesn't depend on typesafe-sdk; an unmodified TypeSafe client also works against Endor by changing base_url and model.

Typed responses. Subclass SystemOneResponse to get one attribute per question:

from endor import ChoiceAnswer, NoulAnswer, SystemOneResponse

class Triage(SystemOneResponse):
    department: ChoiceAnswer
    urgent: NoulAnswer

triage = client.system_one(state, questions, model="tickets", response_model=Triage)
triage.department.choice, triage.urgent.noul

The catalog. client.models.list() returns every name you can pass as model (GET /v1/models): "<project>" and "<project>/base" for each project with a base model, then "<project>/<name>" for each saved model (.kind is project, base or model). client.base_models() returns the base models (GET /v1/base_models) with their option limits, hf_repo, contract and prices (price_per_mtok_decide, price_per_mtok_decide_continuous_learning for managed projects, price_per_gpu_hour). Call a base model through a project: "<project>/base".

Async. await client.system_one_async(...) and await client.models.list_async(); async with EndorClient() closes the connections.

Projects

Everything you create lives in a project, keyed by its name. Each project has a kind and a base model, both set at creation and never changed:

A custom project holds datasets, runs, the models those runs save (named <project>/<name>), and evaluations:

project = client.projects.get_or_create("tickets", base_model="decider-2b")    # kind="custom"
project.datasets.upload("train", endor.data.load_rows("train.jsonl"))   # rows: see "Data format" below
project.datasets.upload("heldout", endor.data.load_rows("heldout.jsonl"))
baseline = project.evaluate("base", "heldout").result()   # names resolve in the project: "base" is "tickets/base"

project.info_ has kind, base_model and paused (managed projects only). project.update(description=...) changes the description. project.models.list() starts with the base model (name == "base"); each saved model has loss, its training loss when it was saved. Runs always train on the project's base model: runs.create() can leave base_model out, and another base raises UnprocessableEntityError. get_or_create raises ValueError when the existing project has another kind or base model.

Managed projects. Endor trains new versions from the project's decisions, named YYYY-MM-DD-N, and "tickets" serves the newest one. The training pipeline is coming soon: until then a managed project serves its base model. Its decisions are billed at the managed price (50% more) from the start.

project = client.projects.create("triage", kind="managed", base_model="decider-2b")
project.update(paused=True)    # stops learning; keeps serving its newest version, at the managed price

Model names you save are lowercase letters, digits, ., _ or -, starting with a letter or digit, up to 63 characters, and can't be base. save_checkpoint raises ValueError for any other name before sending it.

A dataset row is a decision request with labels:

{"state": {"body": "I was charged twice"},
 "questions": {"department": {"type": "choice", "criteria": {"billing": null, "technical": null}},
               "urgent": {"type": "noul", "instructions": "Does this need an answer today?"}},
 "labels": {"department": "billing", "urgent": true}}

Labels can be hard ("billing", true, a level index) or soft (a distribution over the options).

Data format

A row has state (text or JSON), questions (named Noul, Choice or Score objects, as system_one takes), and optionally labels (question name → label), id and weight (default 1). Up to 50,000 rows per dataset. Each labeled question becomes one training example; unlabeled questions are skipped.

Every option has an id, used in labels, training targets and answer probabilities: "false"/"true" for a noul, the criteria keys for a choice, "0"…"n-1" (the level index) for a score.

endor.data.load_rows(path) reads .jsonl or a .json array and also accepts single-question rows, {"state", "question", "expected"} or {"state", "question", "label"?, "target"?}, each becoming one question named decision. endor.data.save_rows(path, rows) writes .jsonl.

Fine-tune in a few lines

from endor.recipes import SupervisedConfig, supervised

cfg = SupervisedConfig(project="tickets", model_name="v1")   # base_model="decider-2b" by default
result = supervised.train(cfg, endor.data.load_rows("train.jsonl"))

result.base_metrics["accuracy"], result.final_metrics["accuracy"]   # held-out: "tickets/base", then yours
client.system_one(state, questions, model=result.model)             # "tickets/v1"

The recipe holds out 10% of the rows, scores the untuned base model ("<project>/base"; the project must have base_model as its base, and one without a base gets it) on them, trains one epoch (learning rate 1e-4 with warmup then linear decay, batches of 16), scores the held-out rows during and after training, and saves the final model. Every number appears on the run's dashboard page. SupervisedConfig has the knobs: base model, rank, learning rate and schedule, batch size, epochs, loss, how often to evaluate.

Training costs and limits

• Billing. Training is billed per GPU-hour, one price for every GPU type, from when a run's GPU is requested (start-up and model loading count) until it is released. Decisions are billed per 1M input tokens, per base model. client.base_models() shows the prices; client.usage(starting_on, ending_before) returns hourly decide rows (input tokens) and train rows (GPU-seconds) with their cost.
• Idle timeout. After 15 minutes without calls, a run saves its state and releases its GPU (status idle). The next call restarts it on a new GPU, which takes a little longer (usually under a minute).
• 4 open runs per org. Every run that isn't closed counts, idle ones included. A fifth runs.create raises LimitReachedError (409, code limit_reached), whose message lists the open runs; close one (run.close() or endor runs close RUN_ID). It is not retried.
• Other limits. 7 projects per org, 20 API keys, 50 models per project and 3 kept-warm models per project also raise LimitReachedError once reached. client.whoami().limits lists every limit with its value.
• Close your runs. Use with project.runs.create(...) as run: (or async with): the run is closed when the block ends, also on errors. runs.create(wait=True) closes the run if you interrupt it while it provisions, and the recipes close their run whatever happens. run.close() refuses later calls, lets calls already accepted finish (status closing), then releases the GPU (status closed); it returns at once, or waits with close(wait=True).
• Balance. When your org's balance is used up (a new org starts with none), decisions, new runs, training calls and server-side evaluations raise InsufficientBalanceError (402, code insufficient_balance) until you add credit in the dashboard under Billing. A call the server already accepted still finishes.

Or drive the loop yourself

rows = endor.data.load_rows("train.jsonl")
train, heldout = endor.data.split(rows, holdout=0.1)
train, heldout = endor.data.rows_to_datums(train), endor.data.rows_to_datums(heldout)

with project.runs.create(base_model="decider-2b", rank=16) as run:
    for batch in endor.data.batches(train, 16):
        fb = run.forward_backward(batch)            # gradients accumulate on the trainer
        opt = run.optim_step(learning_rate=1e-4)    # AdamW step, then zero gradients
        out, step = endor.gather(fb, opt)           # submit both, then wait once
    scored = run.forward(heldout).result()          # option probabilities with the current adapter
    metrics = endor.metrics.decision_metrics(scored.probabilities, [d.target for d in heldout])
    run.log({"heldout/accuracy": metrics["accuracy"]})   # at the run's current step
    model = run.save_checkpoint("v2", include_optimizer=True).result()   # "tickets/v2", usable at once

resumed = project.runs.create(from_model="v2", include_optimizer=True)   # continue exactly where v2 stopped

A run created with from_model inherits that model's base model and LoRA settings (rank, alpha, which layers are trained); the SDK sends only the settings you pass explicitly, and one that conflicts with the model is a 422. A fresh run defaults to rank 16, alpha 32, attention and MLP layers, no readout.

• A Datum is one state, one question, a Target (a hard label or soft probs) and a weight. Each labeled question of a row becomes one datum.
• forward, forward_backward, optim_step and save_checkpoint return futures. Call .result(), await them, or wait for several with endor.gather(...). Each has an _async variant that submits without blocking. A future can stay pending while a GPU starts (usually under a minute); .result() keeps waiting.
• Gradients are the weighted mean over every datum since the last optim_step, so splitting a batch across calls changes nothing. optim_step with nothing accumulated fails with no_gradients. Batches over 1,024 datums are split for you; every datum is checked locally before any chunk is sent, so a bad datum never leaves half a batch applied.
• Losses: cross_entropy (hard or soft targets) and brier.
• optim_step takes AdamParams (betas, epsilon, weight decay, gradient clipping) and the learning rate, which you pass every step: the schedule is yours.
• run.forward scores data with the current adapter without saving a model. project.evaluate(model, dataset) scores a saved model on a stored dataset server-side.
• run.close() releases the trainer (a with block does it for you). Unsaved progress is lost; the SDK warns when you close with optimizer steps newer than your last save.
• Creating a run provisions a trainer, usually in under a minute. runs.create(wait=True) (the default) blocks and logs progress on the endor logger; with wait=False the run returns at once and run.ready is the future.
• If the trainer's GPU dies, pending calls fail with trainer_lost and the run becomes failed (run.info().failure says why). Start a new run with from_model= your last saved model.

Teacher distillation

Label unlabeled rows with a teacher of your choice, then fine-tune on its answer distributions:

from endor.recipes import DistillConfig, SupervisedConfig, distill

class MyTeacher:
    def label(self, rows):                        # one {question name: label} per row
        return [{"department": {"billing": 0.7, "technical": 0.3, "sales": 0.0}} for _ in rows]

cfg = DistillConfig(SupervisedConfig(project="tickets", model_name="distilled"), teacher=MyTeacher())
result = distill.distill(cfg, unlabeled_rows, eval_rows=gold_rows)

Download a trained model

A saved model is a LoRA adapter (PEFT) for its base model. Download its files to run it yourself:

files = project.models.download("v1", "./tickets-v1")   # include_optimizer=True adds optimizer.pt

This writes adapter_model.safetensors, adapter_config.json, endor_manifest.json and, if the model has one, readout.safetensors into the folder. optimizer.pt (Adam state) is only needed to resume training. Each file comes straight from storage over a link that expires after 10 minutes, and is checked against its size and SHA-256 (when the API gives one); a file that fails raises DownloadError and is not left behind. The links are never returned or logged, and your API key is never sent to storage. From the shell: endor models download tickets/v1 -o ./tickets-v1.

Errors and retries

Every API error is an endor.APIError subclass named after its status (NotFoundError, ConflictError, UnprocessableEntityError, RateLimitError, ...) with the server's stable code (unknown_model, invalid_datum, seq_conflict, ...), the offending param when there is one, and the request_id. InsufficientBalanceError (402, insufficient_balance) means your org's balance is used up: add credit in the dashboard. A few codes have their own subclass: LimitReachedError (409 limit_reached) and NoBaseModelError (409 no_base_model), both ConflictErrors, and ModelRequiresProjectError (422 model_requires_project), an UnprocessableEntityError. A training operation that fails on the trainer raises OperationFailedError from .result(), with codes such as oom, no_gradients or trainer_lost. Connection problems raise APIConnectionError and APITimeoutError.

Requests that fail with a connection error, a timeout, a 408, a 429 or a 5xx are retried with exponential backoff and Retry-After, except limit_reached and insufficient_balance, which waiting doesn't fix. Configure it with RetryPolicy:

client = endor.EndorClient(retry=endor.RetryPolicy(max_retries=5, timeout=120))
client = endor.EndorClient(retry=endor.RetryPolicy(max_retries=0))      # no retries

Retries are safe: creates carry an Idempotency-Key, and a run's compute calls carry a sequence number the server checks, so a retried request never runs twice.

Configuration

EndorClient also takes headers for every request, retry, gzip=False to stop compressing large bodies, capture=False to stop sending each run's settings and your git commit hash to the dashboard, and your own httpx client or transport.

What the SDK sends

Besides your API key and the request itself, every request carries:

• User-Agent: endor-python/<version> (python <version>; <os>; <cpu>), with ; cli added for the endor command;
• X-Endor-SDK: endor-python, X-Endor-SDK-Version: <version>, X-Endor-Runtime: python/<version> (<os>; <cpu>) and X-Endor-SDK-Interface: python or cli;
• X-Endor-SDK-Method (the SDK call, e.g. run.forward_backward), X-Endor-SDK-Recipe (inside a recipe), X-Endor-Client-Request-Id (the same on every retry of one call) and X-Endor-Retry-Count (on retries).

runs.create also sends the run's settings and, unless capture=False, your git commit hash and a dirty flag (never file contents).

For every request, the API stores the request id, your org, the calling key, the route, the status and the latency, plus a client record with one field per header: user_agent, sdk, sdk_version, runtime, interface, method, recipe, client_request_id and retry_count (each cut to 200 characters). It also keeps the request body, and the response body when it is JSON and at most 256 KB, in object storage (S3). The secret API key is never stored. You can add your own headers with EndorClient(headers=...); the ones above can't be overridden.

CLI

endor whoami
endor base-models
endor projects list | create NAME --base-model BASE [--kind custom|managed] | delete NAME
endor datasets list PROJECT | upload PROJECT NAME FILE | delete PROJECT NAME
endor runs list PROJECT | show RUN_ID | close RUN_ID
endor models list PROJECT | info MODEL | download MODEL [-o DIR] [--include-optimizer] | set-ttl MODEL SECONDS|none | delete MODEL
endor eval PROJECT MODEL DATASET
endor usage --start 2026-10-01 --end 2026-10-06 [--project P] [--csv]

MODEL is <project>/<name> (<project>/base for the base model). Every command takes -f json. Training itself happens in Python. endor runs close frees a slot when a script left a run open. usage --csv columns are hour, kind, project, base_model, model, training_run_id, input_tokens, gpu_seconds, continuous_learning, price_per_mtok, base_cost_usd, continuous_learning_cost_usd, cost_usd; continuous_learning is true for a managed project's decisions.

Reference

Every public member has a docstring with the details.

License

MIT