TriePick

Turn your LLM into an efficient decision model.

Most agent decisions are small: choose a route, pick the next action, select a tool, classify a state.

You probably don't need a separate decision model architecture. TriePick takes the LLM you already run with vLLM, scores freeform candidate decisions directly, and returns the winner with candidate probabilities.

Quick start

from vllm import LLM
from triepick import TriePick

llm = LLM(
    model="Qwen/Qwen3.5-4B",
)

tp = TriePick(
    llm=llm,
)

result = tp.choose(
    state=(
        "The system is routing requests to the wrong category. "
        "What is the most likely cause?"
    ),
    candidates=[
        "misconfiguration",
        "misclassification",
        "bad training data",
        "outdated routing rules",
    ],
)

print(result.candidate)
print(result.probabilities)

TriePick wraps the exact llm instance you give it; model loading and vLLM configuration stay yours.

How it works

Most multiple-choice approaches turn answers into labels like A, B, C, D.

TriePick scores the actual candidate strings.

It builds a token trie and evaluates only the next-token branches needed to distinguish the candidates. Shared prefixes are reused, and scoring stops once a candidate is uniquely identified.

All scoring rows are flattened into one batched llm.generate() call.

For example, misconfiguration and misclassification share the token branch mis, so TriePick evaluates that shared branch once and only scores the next branch needed to distinguish them.

Batch decisions

TriePick can score multiple independent decisions together.

batch = tp.choose_batch([
    {
        "state": "The payment failed because the card expired.",
        "candidates": [
            "retry payment",
            "request new payment method",
            "cancel account",
        ],
    },
    {
        "state": "A support request contains a billing dispute.",
        "candidates": [
            "billing",
            "technical support",
            "sales",
        ],
    },
])

Decision points across the batch are flattened together before inference and scored in a single vLLM call.

print("generate_calls:", batch.generate_calls)
print("scoring_rows:", batch.scoring_rows)
print("total_candidates:", batch.total_candidates)
print("inference_ms:", round(batch.inference_ms, 2))

Jev / System One

LLMs are great System One models.

TriePick includes a Jev-compatible adapter for System One-style decision APIs.

from triepick.adapters import JevAdapter

jev = JevAdapter(tp)

The adapter supports Jev-style noul, choice, and score operations while using the same TriePick scoring and batching path.

Multimodal

If your vLLM model supports image inputs, TriePick can score decisions using images directly.

result = tp.choose(
    state="Which category best describes this image?",
    candidates=[
        "document",
        "screenshot",
        "photograph",
    ],
    image=image,
)

Images stay in-process and are passed directly to vLLM.

Results

choose() returns a ChoiceResult containing the selected candidate and candidate probabilities.

result.candidate
result.probabilities

It also exposes scoring diagnostics:

result.decision_nodes
result.scoring_rows
result.inference_ms
result.generate_calls

Tie information is available too:

result.is_tie
result.tied_candidates
result.margin

On an exact tie, is_tie is True and tied_candidates contains the tied candidates.

Benchmark

On all 231 redistributable JevBench public decisions with Qwen/Qwen3.5-4B:

Path	Accuracy	Median latency
Native TriePick	73.6%	16.5 ms
JevAdapter	73.6%	17.7 ms

Native TriePick and JevAdapter agreed on 99.6% of predictions.

The benchmark used an NVIDIA RTX PRO 6000 Blackwell Server Edition with vLLM 0.30.1rc1.dev190+g4bb804cc5, Model Runner V2, and Fastokens.

This is a reproducible public-set evaluation, not an official sealed Benchmark Heaven leaderboard submission.

Full benchmark details

Compatibility

Currently tested with:

Model	vLLM	Runner
Qwen/Qwen3.5-4B	0.30.1rc1.dev190+g4bb804cc5	Model Runner V2

TriePick is not tied to Qwen.

At the scoring layer, it needs a supplied vLLM instance with the capabilities required for targeted token scoring, especially logprob_token_ids.

Model Runner V2 is the currently tested path. TriePick does not select, configure, or force a particular model runner.

Installation

From a local checkout:

pip install .

Or install without changing the surrounding vLLM environment:

pip install --no-deps .

Then pass your existing vllm.LLM instance to TriePick:

from triepick import TriePick

tp = TriePick(
    llm=llm,
)

Tutorials

• Quick Start — install TriePick, load Qwen3.5-4B, and make your first decision
• Batching and Multimodal — batch decisions efficiently and score image-based inputs
• Jev / System One — use noul, choice, score, and mixed System One requests

The core idea

A decision model usually does not need to generate every answer.

TriePick scores the token branches that distinguish the candidates and batches those evaluations into one model call.