Endor Python SDK
Decision models that learn from your decisions, and custom models you train yourself.
A decision model takes a state (text or JSON) and one or more named questions, and answers each with a
probability for every option in a single forward pass. No text is generated, so there is no sampling and no
temperature. Three question types cover most decisions:
| Type | Answer |
|---|---|
Noul |
yes or no, as P(true) |
Choice |
one of up to 255 named options, with a probability for each |
Score |
one of 2 to 10 ordered levels, with a probability for each and the expected level |
The usual way to use Endor is a continuous learning project: you pick a base model, send it decisions, tell it when it was wrong, and it gets better at your task by itself. For full control, you can also train custom models with the SDK (see "Advanced: custom models").
Install
pip install endor
export ENDOR_API_KEY=edk_...
Python 3.10 and up. Dependencies: httpx and pydantic.
The SDK calls production (https://api.endor.thetokencompany.com). For another environment, such as staging, set
ENDOR_BASE_URL=https://staging.api.endor.thetokencompany.com or pass EndorClient(base_url=...).
API keys belong to an organization (edk_..., created in the dashboard); everything you create belongs to that org.
Continuous learning projects
Create a project on a base model, then send it decisions. It answers at once with the base model and learns from its own decisions and your corrections, making new versions that you promote to production (or that go there by themselves with auto-promote on):
import endor
from endor import Choice, Noul, Score
client = endor.EndorClient()
project = client.projects.get_or_create("tickets", base_model="mapika/decider-2b") # the base model is chosen once
res = client.system_one(
{"subject": "Charged twice", "body": "I was billed two times for my March invoice."},
{
"department": Choice(instructions="Which team should handle `body`?",
criteria={"billing": "Payments, refunds", "technical": "Bugs, outages", "sales": None}),
"urgent": Noul(instructions="Does the customer need an answer today?"),
"frustration": Score(instructions="How frustrated is the customer?", criteria=["Calm", "Annoyed", "Angry"]),
},
model="tickets", # the project's production version
)
res.choices["department"].choice # "billing"
res.choices["department"].probabilities # {"billing": 0.91, "technical": 0.07, "sales": 0.02}
res.nouls["urgent"].noul # 0.83
res.scores["frustration"].score # 1.4 (expected level), plus .probabilities and .legend
res.model, res.usage.input_tokens # what answered: "tickets:v4", or "tickets" while its base model answers
res.id # the decision's id ("req_..."), to correct it later
Production and preview, like deployments. A project has one production version. A new version lands in the preview and waits there until you promote it to production. Turn auto-promote on (it is off by default) and every new version goes to production as soon as it is ready instead:
model |
Answers with |
|---|---|
"tickets" (project.ref) |
the production version (project.production); the base model until you promote one |
"tickets:preview" (project.preview_ref) |
the preview (project.preview), the newest version while it isn't in production. Without one (always while auto-promote is on) it raises NoPreviewError, and project.preview_ref is None |
"router-2" |
a custom model in production (see "Advanced: custom models") |
Promote any version yourself, for example to roll back:
project.versions() # v1, v2, ... newest first: .production, .preview (never both), .accuracy (%), .loss
project.promote("v3") # v3 to production; a version older than the newest turns auto-promote off
project.set_auto_promote(True) # the newest goes to production at once, and every new version after it (off by default)
project.info() # .production, .preview, .auto_promote, .promoted_at, .versions, .base_model
project.production, project.preview, project.auto_promote # the same, as last fetched
project.download("v3", "./tickets-v3") # a version's LoRA adapter (PEFT), to run it yourself
Older versions can't be called directly ("tickets:v3" raises VersionNotCallableError): promote one to serve it.
Its learning. project.learning() shows how it is going:
learning = project.learning()
learning.status, learning.progress # "learning", 0.4 of the way to the next version
learning.serving # "tickets:v2", or "tickets" while its base model answers
for t in learning.trainings: # newest first: kept or not, with its accuracy and loss
print(t.version, t.kept, t.decisions, t.accuracy, t.against_accuracy)
learning.drift # per question, whether recent traffic changed
A version's and a training's accuracy is in percent (87.5); custom models and evaluations give accuracy from 0 to
1 (0.875).
A project's decisions are billed at the base model's continuous-learning price (client.base_models():
price_per_mtok_decide_continuous_learning). Continuous learning is turned on per organization:
client.whoami().continuous_learning says whether yours has it (while it doesn't, projects and their decisions raise
PermissionDeniedError, 403 continuous_learning_not_enabled). An organization has up to 5 projects
(client.whoami().limits). Names
are lowercase letters, digits, ., _ or -, up to 63 characters, unique in the organization together with your
custom models. client.projects.list(), client.projects.get(name), project.update(description=...) and
project.delete() do what they say.
Corrections
When you learn whether a decision was right, correct it. For each question you checked, say whether the model answered right and, if not, the right answer. The project learns from your corrections in its next versions:
client.corrections.create(res, {
"frustration": {"correct": True}, # the model was right
"department": {"correct": False, "answer": "technical"}, # wrong: the right answer
"urgent": {"correct": False}, # wrong: a yes/no question, so the other answer
}, comment="a bug report, not a billing question")
- An
answermust be exactly one of the question's options as the decision asked it: a choice's option id (case and spelling included), a noul'sTrueorFalse, a score's level index as anint. Anything else, an "other" answer included, raisesUnprocessableEntityError(422,paramquestions.<name>.answer). When none of the options fits, send{"correct": False}. {"correct": False}alone names the right answer for a noul, or a choice or score of two options. With more options the correction'sanswerisNone: only known to be wrong.- A decision has one correction: creating it again replaces it whole.
client.corrections.get(res)reads it,client.corrections.delete(res)withdraws it, andclient.corrections.batch([...])sends up to 1,000 at once, all or nothing ({"request_id", "questions", "comment"?}each), for example to backfill decisions you checked earlier. Each has an_asyncvariant. A decision is itsSystemOneResponseor its id (res.id). - Corrections are free; only continuous learning projects learn from them.
- Endor records a decision just after it answers, so a correction sent at once can find no decision yet.
createandbatchthen try again for up to 5 seconds before they raiseNotFoundError(404not_found), as they do for a decision id Endor doesn't know.
Decisions
model is "<project>", "<project>:preview" (while the project has a preview), the name of a custom model in
production, or a base model's id. There is no default model: pass model= or set one with
EndorClient(model="tickets") or ENDOR_DEFAULT_MODEL, else system_one raises EndorError before sending anything.
Base models directly. Call a base model by its id, owner/model as on Hugging Face (client.base_models() lists
them), at its listed price, price_per_mtok_decide:
client.system_one(state, questions, model="mapika/decider-2b")
Questions can also be plain dicts in the same shape. Every question is answered independently against the same
state. Refer to parts of the state with backticked paths, as in "Is `body` angry?".
Decisions are TypeSafe's API (POST /v1/systemone). The SDK has its own Noul, Choice, Score, answer and
SystemOneResponse types, which send and read exactly TypeSafe's wire format, so it doesn't depend on
typesafe-sdk; an unmodified TypeSafe client also works against Endor by changing base_url and model.
Typed responses. Subclass SystemOneResponse to get one attribute per question:
from endor import ChoiceAnswer, NoulAnswer, SystemOneResponse
class Triage(SystemOneResponse):
department: ChoiceAnswer
urgent: NoulAnswer
triage = client.system_one(state, questions, model="tickets", response_model=Triage)
triage.department.choice, triage.urgent.noul
Benchmarks. Pass exclude_from_training=True to system_one (or EndorClient(exclude_from_training=True)
for every call) to keep decisions out of a project's learning, for example a benchmark or an evaluation. They are
still answered, billed and logged.
Images. On a base model that reads images (client.base_models(): modalities includes "image"), put
endor.Image objects anywhere in the state. PNG, JPEG or WebP, at most 8 per decision and 5 MB each; their tokens
are billed like text. Other base models answer 422 unsupported_modality.
state = {"screenshot": endor.Image.from_path("checkout.png"), "goal": "Pay for the order"}
client.system_one(state, {"pay": Noul(instructions="Is the Pay button visible?")}, model="shop")
Image.from_bytes(data, media_type=None) and Image.from_pil(image, format="PNG") (needs Pillow) build one too.
Images are for decisions only: training data (datasets, datums, evaluation and prediction rows) is text and JSON, and
the API refuses images there (422).
The catalog. client.models.list() returns your own names you can pass as model (GET /v1/models): for each
project "<project>" and, while it has a preview, "<project>:preview", then every custom model in production (.kind is
project or custom; .environment is production or preview for a project's names). client.base_models() returns the base models (GET /v1/base_models) with their option limits,
modalities, hf_repo, contract and prices (price_per_mtok_decide for the base itself and custom models,
price_per_mtok_decide_continuous_learning for projects, price_per_gpu_hour for training).
To choose a base model, compare what each is meant for, its ranks, speed, context and price:
for b in client.base_models():
i = b.decision_index # its entry on the Jev Decision Index, None if it isn't there
print(b.id, i and i.meant_for, # ["Code", "Agents", ...], best first; empty when none
i and f"{i.rank}/{i.entrants}", # its rank on the board
b.tokens_per_s, # decision input tokens per second Endor measured, None if not measured
b.max_question_tokens, b.price_per_mtok_decide)
decision_index also has tied (within the board's tie band of another entry), use_case_ranks (its mean rank on
each use case's benchmarks), task_ranks (its rank on each public benchmark, by what it tests, for example
"phishing detection"), note (where the index ran another form of the weights than Endor serves) and url (its
scores on the index). meant_for holds "Code", "Agents", "Documents and email" and "Finance" where its mean
rank on that use case is in the board's top quarter. endor base-models prints the same as a table.
Many rows at once. client.predict(model, rows) returns the answers to every question of every row, in the rows'
order, from one server-side evaluation of them: nothing to upload, nothing scored, billed as the same decisions.
model is a custom model's name (in production or archived) or a base model's id. Rows are {"state", "questions"} (any shape
client.datasets.upload takes; labels are ignored), at most 50,000 and 32 MB per call. While it runs it prints a
progress line every 10 seconds (progress=False turns it off); timeout= bounds the wait. An unknown model raises
NotFoundError (404 unknown_model).
answers = client.predict("router-2", [{"state": t, "questions": questions} for t in new_tickets])
answers[0]["department"].choice # "billing": each row's answers as system_one(...).answers has them
Async. await client.system_one_async(...), await client.predict_async(...) and
await client.models.list_async(); async with EndorClient() closes the connections. One client works across
several asyncio.run() calls: each event loop gets its own connections.
Data format
Datasets, evaluations and predictions take decision rows. A row has state (text or JSON), questions (named
Noul, Choice or Score objects, as system_one takes), and optionally labels (question name → label), id and
weight (default 1). Up to 50,000 rows per dataset. Each labeled question becomes one training example; unlabeled
questions are skipped.
{"state": {"body": "I was charged twice"},
"questions": {"department": {"type": "choice", "criteria": {"billing": null, "technical": null}},
"urgent": {"type": "noul", "instructions": "Does this need an answer today?"}},
"labels": {"department": "billing", "urgent": true}}
Every option has an id, used in labels, training targets and answer probabilities: "false"/"true" for a noul,
the criteria keys for a choice, "0"…"n-1" (the level index) for a score.
| Type | Hard label | Soft label (every option, non-negative, summing to 1 ± 0.02) |
|---|---|---|
noul |
true / false |
0.8 (P(true)) or {"true": 0.8, "false": 0.2} |
choice |
"billing" |
{"billing": 0.7, "technical": 0.2, "sales": 0.1} |
score |
2 (level index) or "2" (its option id) |
[0.1, 0.6, 0.3] or {"0": 0.1, "1": 0.6, "2": 0.3} |
A soft label is the probability map itself. {"probs": {...}} is the shape of a Datum target (below), not of a
row label, and is refused with a ValueError that says so.
endor.data.load_rows(path) reads .jsonl or a .json array and also accepts single-question rows,
{"state", "question", "expected"} or {"state", "question", "label"?, "target"?}, each becoming one question named
decision. endor.data.save_rows(path, rows) writes .jsonl.
Advanced: custom models
A custom model is one you train yourself with the SDK, on a base model, from your own labeled data. Each save is a new model with its own name, unique in the organization together with your projects. A saved model starts in the archive: it can be downloaded, evaluated and trained from, and decisions can call it by its name once you promote it to production:
with client.custom_models.create("router-2", base_model="mapika/decider-2b") as run: # a run that saves as "router-2"
... # forward_backward, optim_step (below), or use the recipe
run.save_checkpoint().result() # "router-2", in the archive
client.custom_models.promote("router-2") # production: model="router-2" answers
client.system_one(state, questions, model="router-2")
client.custom_models.archive("router-2") # back to the archive: decisions refuse it
client.custom_models.list()lists your custom models, the ones in production first (production=Trueorproduction=Falsekeeps one of the two);get(name)hasstate("production"or"archived"),production,promoted_at,base_model,step, andlossandaccuracy(0 to 1): the run's training loss and accuracy at the model's step.- A model in production has
serving:"deploying"right afterpromote(decisions are accepted; the first ones may wait while a decision server loads it), then"live", or"failed"withserving_errorsaying why (promote(name)again retries). It is None while archived.endor custom-models listandinfoshow it. - An organization has a few models in production at once (
max_production_models_per_orginclient.whoami().limits). When every slot is taken,promoteraisesProductionFullErrornaming the models in production;promote(name, replace="router-1")archives that one in the same step.archive(name)frees a slot. Ask us for more production slots. - Decisions refuse an archived model with
ModelNotInProductionError(409model_not_in_production). Models in production never expire:set_ttl(name, seconds)gives an archived model a lifetime, anddelete(name)removes an archived one (ModelInProductionErrorfor one in production: archive it first). - Datasets (
client.datasets), runs (client.runs) and evaluations (client.evaluations) belong to the organization.client.runs.create(base_model, model="router-2", ...)is the same asclient.custom_models.create("router-2", base_model, ...).
client.datasets.upload("train", endor.data.load_rows("train.jsonl")) # rows: see "Data format"
client.datasets.upload("heldout", endor.data.load_rows("heldout.jsonl"))
client.evaluations.create("mapika/decider-2b", "heldout").result() # the untuned base, as a baseline
client.evaluations.create("router-2", "heldout").result() # a custom model, in production or not
client.evaluations.create("router-2", rows=more_rows).result() # or rows sent with the call, no dataset
An evaluation scores a row's labeled questions; a row without labels is answered on every question and not scored
(its answers are in the results' rows_url, which client.predict reads for you). Its accuracies are from 0 to 1.
client.evaluations.get(evaluation_id) fetches one again, with rows_done of rows_total while it runs, and
client.evaluations.list(run_id=None, model=None) lists them. An unknown model raises NotFoundError (404
unknown_model).
Model names are lowercase letters, digits, ., _ or -, starting with a letter or digit, up to 63 characters,
and can't be a base model id. save_checkpoint raises ValueError for a bad name before sending it. Each name is
saved once: a name a custom model or project already has raises ConflictError (409 conflict) at once. The run
itself is unaffected, but an error you don't catch leaves the with block, which closes the run. For checkpoints
during a long run, give each its own name, e.g. run.save_checkpoint(f"router-ckpt-{step}", ttl_seconds=86_400).
Train in a few lines
from endor.recipes import SupervisedConfig, supervised
cfg = SupervisedConfig(model_name="router-2") # base_model="mapika/decider-2b" by default
result = supervised.train(cfg, endor.data.load_rows("train.jsonl"))
result.base_metrics["accuracy"], result.final_metrics["accuracy"] # held-out: the untuned base, then yours
client.custom_models.promote(result.model) # to production
client.system_one(state, questions, model=result.model) # "router-2"
The recipe holds out 10% of the rows, scores the untuned base model on them, trains one epoch (learning rate 1e-4
with warmup then linear decay, batches of 16), scores the held-out rows during and after training, and saves the
final adapter as the custom model model_name, in the archive. Every number appears on the run's dashboard page.
SupervisedConfig has the knobs: base model, rank, learning rate and schedule, batch size, epochs, loss, how often to
evaluate. The recipe sets the run's total_steps from the data size and epochs, so the dashboard shows a progress bar
with an ETA, and it prints a progress line (step 37/120 (31%) · loss 0.4120 · ~3 min left; progress=False turns it
off). It sends each step before the last one's results are read (steps_in_flight=2), so the trainer doesn't wait for
a round trip between steps; results, metrics and evaluations are the same as with steps_in_flight=1.
Before training, the recipe prints a warning for each question with options that have no training labels, or fewer
than 10 (warning: question 'dept': no training labels for refund; the model can't learn an option it never sees).
Add rows for those options, or drop them from the question.
Training costs and limits
- Billing. Training is billed by the second for the GPU time your training calls use, one price per GPU-hour for
every base model; start-up, model loading and the time between calls are free, and each run session costs at least
60 GPU-seconds. Decisions are billed per 1M input tokens, per base model.
client.base_models()shows the prices;client.usage(starting_on, ending_before)returns hourlydeciderows (input tokens) andtrainrows (GPU-seconds) with their cost. - Idle timeout. After 2 minutes without calls, a run saves its state and frees its GPU (status
idle). The next call resumes it in a few seconds. While a run is open, the SDK sends a heartbeat every minute from a background thread, so the run stays open however long you wait between calls. When the process ended without closing the run, Endor closes it an hour later; saved models are kept. To continue it,client.runs.get(run_id)gives the run again (run.info_.stepis its step), and that handle sends the heartbeat too until the run is closed or the handle is dropped. The handlesclient.runs.list()returns are for looking: they send none. - Open runs per org. Every run that isn't closed counts toward the org's open-run limit, idle ones included: 25
by default, and
max_active_runsinclient.whoami().limitsis your org's own (endor whoamishows it). One moreruns.createraisesLimitReachedError(409, codelimit_reached), whose message lists the open runs; close one (run.close()orendor runs close RUN_ID). It is not retried. - Other limits. Projects, custom models (and those in production), datasets and API keys also have limits (
LimitReachedErroronce reached).client.whoami().limitslists every limit with its value. - Close your runs. Use
with client.runs.create(...) as run:(orasync with): the run is closed when the block ends, also on errors.runs.create(wait=True)closes the run if you interrupt it while it provisions, and the recipes close their run whatever happens.run.close()refuses later calls, lets calls already accepted finish (statusclosing), then releases the GPU (statusclosed); it returns at once, or waits withclose(wait=True). - Balance. When your org's balance is used up (a new org starts with none), decisions, new runs, training calls
and server-side evaluations raise
InsufficientBalanceError(402, codeinsufficient_balance) until you add credit in the dashboard under Billing. A call the server already accepted still finishes.client.balance()(orendor whoami) shows what is left, in USD.
Or drive the loop yourself
rows = endor.data.load_rows("train.jsonl")
train, heldout = endor.data.split(rows, holdout=0.1)
train, heldout = endor.data.rows_to_datums(train), endor.data.rows_to_datums(heldout)
with client.custom_models.create("router-2", base_model="mapika/decider-2b", rank=16) as run:
steps = endor.Steps() # keeps 2 steps in flight
for batch in endor.data.batches(train, 16):
fb = run.forward_backward(batch) # gradients accumulate on the trainer
opt = run.optim_step(learning_rate=1e-4) # AdamW step, then zero gradients
for out, step in steps.add(fb, opt): # sent; returns the previous step's results
print(step.step, out.loss)
for out, step in steps.wait(): # the last step
print(step.step, out.loss)
scored = run.forward(heldout).result() # option probabilities with the current adapter
metrics = endor.metrics.decision_metrics(scored.probabilities, [d.target for d in heldout])
run.log({"heldout/accuracy": metrics["accuracy"]}) # at the run's current step
model = run.save_checkpoint(include_optimizer=True).result() # "router-2", in the archive
resumed = client.custom_models.create("router-3", from_model="router-2", include_optimizer=True) # continue exactly
A run created with from_model (a custom model's name) inherits that model's base model and LoRA settings (rank,
alpha, which layers are trained); the SDK sends only the settings you pass explicitly, and one that conflicts with the
model is a 422. A fresh run defaults to rank 16, alpha 32, attention and MLP layers, no readout.
- A
Datumis one state, one question, aTarget(a hardlabelor softprobs) and a weight. Each labeled question of a row becomes one datum. A target isTarget(label="billing")orTarget(probs={"billing": 0.7, "tech": 0.3})(as a dict,{"probs": {...}}); the bare map is a row label's shape and is refused.endor.data.label_target(question, label)turns a row label into aTarget. forward,forward_backward,optim_stepandsave_checkpointreturn futures. Call.result(),awaitthem, or wait for several withendor.gather(...). Each has an_asyncvariant that submits without blocking. A future can stay pending while a GPU starts (usually under a minute);.result()keeps waiting.endor.Steps()sends each step before the last one's results are read, so the trainer starts the next step as soon as it finishes one instead of waiting for your round trip (most of a second per step).steps.add(fb, opt)returns the results of the steps it waited for (the step before, once two are out), oldest first, andsteps.wait()the rest; a failed step raises from the call that reads it. A call you send between steps (run.forward,save_checkpoint) runs after every step added so far.Steps(in_flight=1)waits for each step, likeendor.gather(fb, opt)after every step.add_asyncandwait_asyncare for async code.- Gradients are the weighted mean over every datum since the last
optim_step, so splitting a batch across calls changes nothing.optim_stepwith nothing accumulated fails withno_gradients. Batches over 1,024 datums are split for you; every datum is checked locally before any chunk is sent, so a bad datum never leaves half a batch applied. - Losses:
cross_entropy(hard or soft targets) andbrier. optim_steptakesAdamParams(betas, epsilon, weight decay, gradient clipping) and the learning rate, which you pass every step: the schedule is yours.run.predict(rows)answers decision rows with the current adapter before you save it: a future of one{name: answer}per row, in order, asclient.system_oneanswers them (labels are ignored). It runs on the training GPU in order with the run's other calls and is billed as training GPU time, likeforward. Lists over 1,024 questions are split for you. Text and JSON states only.run.forwardscores data with the current adapter without saving a model.client.evaluations.create(model, dataset)scores a saved model server-side (or onrows=sent with the call).run.close()releases the trainer (awithblock does it for you). Unsaved progress is lost; the SDK warns once when you close with optimizer steps newer than your last save.- Creating a run provisions a trainer: under a minute on a warm GPU, a few minutes to start a cold one.
runs.create(wait=True)(the default) blocks and logs progress at INFO on theendorlogger. Since Python hides INFO logs unless you configure logging, it also prints one line to stderr once the wait passes 10 seconds (run run_...: waiting for a GPU (starting a cold one can take a few minutes)…);runs.create(progress=False)(orcustom_models.create(..., progress=False),SupervisedConfig(progress=False)) leaves it out. Withwait=Falsethe run returns at once andrun.readyis the future. - If the trainer's GPU dies, pending calls fail with
trainer_lostand the run becomesfailed(run.info().failuresays why). Start a new run withfrom_model=your last saved model. - Progress: pass
runs.create(total_steps=N)(or laterrun.set_total_steps(N)) and the dashboard shows a progress bar with an ETA; without it, the step count and steps per minute.run.info()hasstep,total_steps,seconds_per_step(median over the last 20 steps),progress(0 to 1) andeta_seconds.
Weights & Biases
Endor logs runs to Weights & Biases from its servers; your training code doesn't change. Connect W&B once in the
dashboard (Settings > Integrations, with a W&B API key that Endor keeps in AWS Secrets Manager), and every run logs to
it; turn it off or on per run with client.runs.create(..., wandb=False) or wandb=True. Each run becomes a W&B run
with the same id, with the base model, LoRA settings and your config, and every run metric (train/loss,
train/accuracy, train/grad_norm, train/lr, run.log) against the Endor step; it is finished when the run
closes. run.info().wandb_url links to it.
Teacher distillation
Label unlabeled rows with a teacher of your choice, then fine-tune on its answer distributions:
from endor.recipes import DistillConfig, SupervisedConfig, distill
class MyTeacher:
def label(self, rows): # one {question name: label} per row
return [{"department": {"billing": 0.7, "technical": 0.3, "sales": 0.0}} for _ in rows]
cfg = DistillConfig(SupervisedConfig(model_name="router-distilled"), teacher=MyTeacher())
result = distill.distill(cfg, unlabeled_rows, eval_rows=gold_rows)
Download a trained model
A custom model (in production or archived) or a project's version is a LoRA adapter (PEFT) for its base model. Download its files to run it yourself:
files = client.custom_models.download("router-2", "./router-2") # include_optimizer=True adds optimizer.pt
files = project.download("v3", "./tickets-v3") # a continuous learning project's version
This writes adapter_model.safetensors, adapter_config.json, endor_manifest.json and, if the model has one,
readout.safetensors into the folder. optimizer.pt (Adam state) is only needed to resume training, and only a model
saved with run.save_checkpoint(include_optimizer=True) has it (CustomModelInfo.has_optimizer): for any other model,
or a project's version, include_optimizer=True raises DownloadError before anything is written. Each file comes
straight from storage over a link that expires after 10 minutes, and is checked against its size and SHA-256 (when
the API gives one); a file that fails raises DownloadError and is not left behind. The links are never returned or
logged, and your API key is never sent to storage. From the shell: endor custom-models download router-2 -o ./router-2.
Errors and retries
Every API error is an endor.APIError subclass named after its status (NotFoundError, ConflictError,
UnprocessableEntityError, RateLimitError, ...) with the server's stable code (unknown_model,
invalid_datum, seq_conflict, ...), the offending param when there is one, and the request_id.
InsufficientBalanceError (402, insufficient_balance) means your org's balance is used up: add credit in the
dashboard. A few codes have their own subclass: LimitReachedError (409 limit_reached) and its
ProductionFullError (409 production_full), ModelNotInProductionError (409 model_not_in_production) and
ModelInProductionError (409 model_in_production), all ConflictErrors; and NoPreviewError (404 no_preview) and
VersionNotCallableError (404 version_not_callable), both NotFoundErrors. A training operation that fails on the trainer raises OperationFailedError from .result(), with codes
such as oom, no_gradients or trainer_lost. Connection problems raise APIConnectionError and
APITimeoutError. A successful response whose body doesn't have the expected shape raises ResponseValidationError
(an APIError) naming the endpoint and the field, from every method.
Requests that fail with a connection error, a timeout, a 408, a 429 or a 5xx are retried with exponential backoff
and Retry-After, except limit_reached, production_full and insufficient_balance, which waiting doesn't
fix. Configure it with
RetryPolicy:
client = endor.EndorClient(retry=endor.RetryPolicy(max_retries=5, timeout=120))
client = endor.EndorClient(retry=endor.RetryPolicy(max_retries=0)) # no retries
A model that hasn't been called for a while may start cold: its first decision can wait a few minutes. The SDK
waits up to 330 s for a decision (30 s for other calls). If the model is still not ready after 5 minutes, the API
answers 503 warming_up with Retry-After, and the SDK retries after that wait.
Retries are safe: creates carry an Idempotency-Key, and a run's compute calls carry a sequence number the server
checks, so a retried request never runs twice.
str(error) ends with one line on how to report the problem (below); error.message is the server's message
alone, for showing to your own users.
Reporting problems
If Endor does something you didn't expect, or the docs, an error message or the API are unclear, tell the Endor
team. This is meant for you and for the coding agent you handed the integration to: every error's str() ends with
the line below, so an agent sees it right where it gets stuck.
endor.user_report("what you did, what happened, what you expected", kind="bug") # no client needed
client.user_report("the docs don't say how to label a score question", kind="confusing")
client.user_report("the integration runs: tickets are routed with the tickets project", kind="works")
kind is bug (Endor did something unexpected), confusing (something is unclear) or works (send one once an
integration runs, so we know what worked). A report goes to POST /v1/user_reports with your message, the request it is
about (request_id=, by default the latest failed request of the client, or of any client in the process for
endor.user_report) and a small context: the SDK and Python versions, the platform, and that failure's status, code and
endpoint, plus anything you pass as context={...} (short values). Never your API key, a request body or a
decision's content. It needs your API key (the client's, or ENDOR_API_KEY). From a shell: endor user-report "what happened" --kind bug [--request-id req_...].
If you give a coding agent the Endor integration, add to its instructions: "If Endor behaves unexpectedly or a step
is unclear, report it with endor.user_report(...); report kind="works" once the integration runs."
Configuration
| Argument | Environment variable | Default |
|---|---|---|
api_key |
ENDOR_API_KEY |
required |
base_url |
ENDOR_BASE_URL |
production |
model |
ENDOR_DEFAULT_MODEL |
none: pass model= to system_one, e.g. "tickets" |
timeout |
30 s per request, 330 s for a decision (a future poll also waits up to 25 s on the server) | |
ENDOR_LOG_LEVEL |
unset; info logs one line per request on the endor logger |
|
report_latency |
ENDOR_REPORT_LATENCY |
on; False or 0 stops sending X-Endor-Client-Latency (see below) |
EndorClient also takes headers for every request, retry, gzip=False to stop compressing large bodies, capture=False
to stop sending each run's settings and your git commit hash to the dashboard, and your own httpx client or
transport.
What the SDK sends
Besides your API key and the request itself, every request carries:
User-Agent: endor-python/<version> (python <version>; <os>; <cpu>), with; cliadded for theendorcommand;X-Endor-SDK: endor-python,X-Endor-SDK-Version: <version>,X-Endor-Runtime: python/<version> (<os>; <cpu>)andX-Endor-SDK-Interface: pythonorcli;X-Endor-SDK-Method(the SDK call, e.g.run.forward_backward),X-Endor-SDK-Recipe(inside a recipe),X-Endor-Client-Request-Id(the same on every retry of one call) andX-Endor-Retry-Count(on retries);X-Endor-Client-Latency: <request id>=<ms>: how long the previous request took from sending it to reading the response, network and connection set-up included, sent once with the next request (nothing waits for it), so the dashboard shows the latency you see next to Endor's own.EndorClient(report_latency=False)orENDOR_REPORT_LATENCY=0turns it off.
runs.create also sends the run's settings and, unless capture=False, your git commit hash and a dirty flag
(never file contents). While a run is open, a background thread sends POST /v1/runs/{id}/heartbeat, with no
body, once a minute. A report (endor.user_report) sends only what "Reporting problems" lists.
For every request, the API stores the request id, your org, the calling key, the route, the status, the latency and
the end-to-end time the SDK reported for it, plus a client record with one field per header: user_agent, sdk, sdk_version, runtime, interface,
method, recipe, client_request_id and retry_count (each cut to 200 characters). It also keeps the request
body, and the response body when it is JSON and at most 256 KB, in object storage (S3). The secret API key is never
stored. You can add your own headers with EndorClient(headers=...); the ones above can't be overridden.
CLI
endor whoami
endor base-models
endor projects list | create NAME --base-model BASE | delete NAME
endor projects versions NAME | promote NAME VERSION | auto-promote NAME on|off
endor custom-models list [--production | --archived] | info NAME | promote NAME [--replace OTHER] | archive NAME
endor custom-models download NAME [-o DIR] [--include-optimizer] | set-ttl NAME SECONDS|none | delete NAME
endor datasets list | upload NAME FILE | delete NAME
endor runs list | show RUN_ID | close RUN_ID
endor eval MODEL DATASET
endor usage --start 2026-10-01 --end 2026-10-06 [--project P] [--csv]
endor user-report "what you did, what happened, what you expected" [--kind bug|confusing|works] [--request-id REQ]
Projects are continuous learning projects; custom models are the ones you train with the SDK. MODEL in eval is a
custom model's name or a base model's id (owner/model); eval prints the accuracy overall and per question. endor base-models prints
one line per base model: its Decision Index rank (= marks a tie), tokens/s, context (max_question_tokens), price
per 1M input tokens (called directly, or a custom model on it), and what it is meant for. endor whoami also
shows the org's balance (balance_usd), whether it has continuous learning (continuous_learning) and its limits. Every command takes -f json (or --format json), before or after the command (endor -f json whoami or endor whoami -f json), and needs ENDOR_API_KEY, endor user-report too. Training itself happens in Python. endor runs close
frees a slot when a script left a run open. usage --csv columns are hour, kind, project, base_model, model, training_run_id, input_tokens, gpu_seconds, continuous_learning, price_per_mtok, base_cost_usd, continuous_learning_cost_usd, cost_usd; continuous_learning is true for a continuous learning project's
decisions; a custom model's decisions have no project.
Reference
| Object | Members |
|---|---|
EndorClient(*, api_key, model, retry, timeout, headers, base_url, capture, exclude_from_training, gzip, http_client, async_http_client, transport, async_transport, report_latency) |
system_one(state, questions, *, model, retry, timeout, extra_headers, extra_body, exclude_from_training, response_model), system_one_async, predict(model, rows, *, progress=True, timeout=None) → one {question: answer} per row, predict_async, models.list(), models.list_async(), base_models(), projects, custom_models, datasets, runs, evaluations, corrections, whoami(), balance() → USD, usage(starting_on, ending_before, project=None), user_report(message, *, kind="bug", request_id=None, context=None), user_report_async, close(), aclose(), context managers |
endor.user_report(message, *, kind="bug", request_id=None, context=None, api_key=None, base_url=None, timeout=None) |
endor.user_report_async(...); returns UserReportReceipt (id, message) |
client.projects |
create(name, base_model, description=None), get(name), get_or_create(name, base_model, description=None), list(limit=None, offset=0) (all pages) |
Project |
.name, .info_, .ref ("<project>"), .preview_ref ("<project>:preview", or None without a preview), .production, .preview, .auto_promote, info(), learning() → Learning, versions() → list[VersionInfo], promote(version), set_auto_promote(on), download(version, path, include_optimizer=False), update(*, description), delete() |
client.corrections |
create(request_id, questions, *, comment=None) → CorrectionInfo, batch(items) → list[CorrectionInfo], get(request_id), delete(request_id), each with an _async variant; request_id is a SystemOneResponse or its id |
client.custom_models |
list(production=None, limit=None, offset=0) → list[CustomModelInfo] (production first), get(name), create(name, base_model=None, *, run_name=None, **run_options) → Run, promote(name, replace=None), archive(name), set_ttl(name, ttl_seconds), download(name, path, include_optimizer=False), delete(name) |
client.datasets |
upload(name, rows), list(), get(name), rows(name, page=500), delete(name) |
client.runs |
create(base_model=None, *, model, rank, alpha, seed, train_attn, train_mlp, train_readout, from_model, include_optimizer, name, tags, config, user_metadata, total_steps, wandb, wait, progress) → Run, get(run_id), list(tag=None, limit=None, offset=0) |
client.evaluations |
create(model, dataset=None, run_id=None, *, rows=None) → APIFuture[Evaluation], get(evaluation_id), list(run_id=None, model=None) |
Run |
.id, .model, .info_, .ready, .next_seq_id, forward(data, loss_fn), forward_backward(data, loss_fn), optim_step(adam_params=None, *, learning_rate=None), save_checkpoint(name=None, *, include_optimizer, ttl_seconds, user_metadata) → APIFuture[str], predict(rows) → APIFuture[list[dict[str, Answer]]], the _async variants of those five, close(*, wait=False, timeout=None), close_async(...), info(), set_total_steps(n), log(metrics, step=None), metrics(keys=None, since_step=None), log_eval(model, results, *, step, name), sync and async context manager |
APIFuture[T] |
result(timeout=None), await f, result_async(timeout), done(), info, cancel(), cancel_async(), APIFuture.completed(value); endor.gather(*futures), endor.gather_async(*futures) |
endor.Steps(in_flight=2) |
add(*futures) → the results of the steps it read, wait() → the rest, add_async, wait_async, len(steps) |
endor.data |
to_row, load_rows(path), save_rows(path, rows), label_target(question, label), row_to_datums(row), rows_to_datums(rows), split(rows, holdout=0.1, seed=0), batches(items, size, *, shuffle=True, seed=0) |
endor.metrics |
decision_metrics(probs, targets, bins=10) → {n, accuracy, nll, brier, ece, mean_confidence, selective} |
endor.recipes.supervised |
SupervisedConfig, SupervisedResult, train(cfg, rows, eval_rows=None, client=None), evaluate_run(run, datums), evaluate_model(client, model, rows), lr_at(cfg, step, total) |
endor.recipes.distill |
Teacher (protocol), DistillConfig(supervised, teacher, budget, seed), label_with_teacher(rows, cfg), distill(cfg, unlabeled_rows, eval_rows, client=None) |
| Types | Noul, Choice, Score, NoulCriteria; NoulAnswer, ChoiceAnswer, ScoreAnswer, SystemOneResponse, Usage, ModelMetadata, ListModelsResponse, BaseModelInfo, DecisionIndexInfo; Datum, DecisionRow, Target, LoraConfig, AdamParams, ForwardOutput, OptimStepOutput; CorrectionInfo, QuestionCorrectionInfo, QuestionCorrection, CorrectionAnswer; ProjectInfo, VersionInfo, Learning, Training, Drift, CustomModelInfo, RunInfo, LoraInfo, DatasetInfo, Evaluation, MetricPoint, DownloadedFile, UsageRow, WhoAmI; helpers option_keys, answer_probabilities, question_dict |
| Errors | EndorError; APIError with BadRequestError, AuthenticationError, InsufficientBalanceError, PermissionDeniedError, NotFoundError (NoPreviewError, VersionNotCallableError), ConflictError (LimitReachedError > ProductionFullError, ModelNotInProductionError, ModelInProductionError), PayloadTooLargeError, UnprocessableEntityError, RateLimitError, OverloadedError, InternalServerError, ResponseValidationError; APIConnectionError, APITimeoutError; OperationFailedError |
Every public member has a docstring with the details.
License
MIT
Metadata
Release files for endor 0.9.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| endor-0.9.0.tar.gz | 190.3 kB | With provenance | Details for endor-0.9.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| endor-0.9.0-py3-none-any.whl | Python 3 | none | any | Details for endor-0.9.0-py3-none-any.whl |
Total release size: 300.9 kB
Release files / endor-0.9.0.tar.gz
| Download URL | endor-0.9.0.tar.gz |
|---|---|
| Size | 190.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2ddc6599f2102bfd670afecb49157aa1ef24c7b143fb8bcec0555832fe9d6a4c
|
|
BLAKE2b-256 checksum How to use checksums |
2f6361141aa8c46e75721f222bf16dbd3c8456ea5879484ba753dae76c1fd72f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / endor-0.9.0-py3-none-any.whl
| Download URL | endor-0.9.0-py3-none-any.whl |
|---|---|
| Size | 110.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
019e7417dd01975627d906270fa03f176bc50b88b4f8d6579e0b334b12e2e032
|
|
BLAKE2b-256 checksum How to use checksums |
f7323394933202b94fa5ba3830505804fe3665d73615310f496f69ccf7bb9251
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logSecurity for endor 0.9.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for endor-0.9.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.