typed-encoders

Training recipes for small encoders that answer typed questions about a text: decisions (pick among options, with a probability for each) and extraction (the spans that answer a field, or nothing). Each recipe reproduces a released model.

The data is tasksource/tasksource-jev-typed-decisions (its default config for decisions, its extraction config for extraction), plus the sources listed in each recipe. Extraction and NLI train with tasknet.

pip install typed-encoders                     # the models; for the recipes: pip install -e ".[decider,extraction,nli]"
pip install git+https://github.com/apolinario/decision-index   # decider recipe only (not on PyPI)
bash recipes/decider/decider_nano_v5.sh          # needs a Decision Index kit checkout, see the script
bash recipes/extraction/clairic_extraction.sh
bash recipes/nli/ultrabert_nli.sh

Decider

from typed_encoders.decider import Decider
d = Decider.from_pretrained("tasksource/tasksource-decider-nano")      # a released checkpoint, or a training run's dir
d.score_many([("The battery died after a week.", "What is the sentiment?", ["negative", "positive"])])
# -> [array([ 0.39, -4.19])]  one logit per option, softmax over a question's options

A joint cross-encoder (typed_encoders/decider.py): every option of a question gets a scalar, softmaxed over the question's options. A request's state is encoded once and shared by all its questions and options (tree attention: an option sees the state, its question and itself), so a request with many questions costs little more than one. Trained with soft cross-entropy against target distributions on a 480k-request mix:

Every part is firewalled against the Decision Index 0.3 suite (exact matches and word 6-gram overlap of the content), and benchmarks without a train split are never used. All sources are pinned (download.py), and the manifests rebuild byte-identical to the ones v5 was trained on.

Extraction

A query-conditioned span tagger: the input is field instruction [SEP] text, and one O/B/I head tags the text, so one model handles single values, lists and absent fields, for fields it has never seen. train_packed.py is the one-pass variant: every field of a document in one forward pass (GLiNER-style marker tokens and a field × token head). Evaluations: the extraction test split (eval_jev.py), out-of-domain MRQA and FinEntity (eval_ood.py), and LocalLLaMA/typed-extraction (eval_typed.py, zero-shot).

NLI

Multi-task training on every tasksource NLI-style task with tasknet (shared encoder, one head per label set).