trellar
A lightweight Python client for the Trellar confidence evaluation API. Attach a callback to your LangChain / LangGraph run, then call evaluate_confidence() when you want a score. Context, trace ID, and agent name are picked up automatically — no manual wiring.
This library cannot be used without an API key from trellar.io.
Create an account and API key
trellar.io is the only place that issues API keys for this library. Create an account there, then generate an API key from the dashboard. Without that key, evaluate_confidence() cannot authenticate and the client will not work.
Then pass the key into the SDK (see Environment Variables):
evaluate_confidence(api_key="..."), orTRELLAR_API_KEYin the environment
Installation
pip install "trellar[langchain]"
The langchain extra is required because agent runs are captured via a LangChain callback handler (get_agent_guard). Requires Python 3.9+.
Quick Start
from trellar import get_agent_guard, evaluate_confidence
# agent_name must be a stable, unique name for this agent graph — the
# backend uses it to track the graph's network profile across runs.
guard = get_agent_guard("research-agent")
# call evaluate_confidence() from a node, while the run is still in progress —
# context, trace_id, and agent_name are picked up from the guard automatically
def report_confidence(state):
result = evaluate_confidence()
print(result.score) # int, 1-10
print(result.explanation) # str, human-readable reasoning
return state
graph.invoke(inputs, config={"callbacks": [guard]})
Strands Agents
pip install "trellar[strands]" # requires Python 3.10+
from strands import Agent
from strands.multiagent import GraphBuilder
from trellar import get_strands_guard, evaluate_confidence
guard = get_strands_guard("research-agent")
# Give every agent a stable name, and register the guard on each agent...
searcher = Agent(name="searcher", hooks=[guard])
reporter = Agent(name="reporter", hooks=[guard])
# ...and on the Graph/Swarm, so the whole run is one trace.
builder = GraphBuilder()
builder.add_node(searcher, "searcher")
builder.add_node(reporter, "reporter")
builder.add_edge("searcher", "reporter")
builder.set_entry_point("searcher")
builder.set_hook_providers([guard])
graph = builder.build()
graph("Find me something to report on")
Call evaluate_confidence() from inside the run (a graph node or a tool), exactly as with LangChain. ObservabilityMode works the same way. See orchestrations_examples/our_lab_with_aviran/car_stocks_buy_mcp_celery.py/car_stocks_buy_from_remote_provider_orchestration_strands.py for a full example.
get_strands_single_call_guard
For one Agent called once (no Graph/Swarm). The run is over when the call returns, so use ObservabilityMode.ALWAYS (or IF_NOT_EVALUATED) and read the result off the guard:
from strands import Agent
from trellar import get_strands_single_call_guard, ObservabilityMode
guard = get_strands_single_call_guard("faq-agent", ObservabilityMode.ALWAYS)
agent = Agent(name="faq_agent", hooks=[guard])
agent("What time does the office open?")
result = guard.trellar_evaluate_result # AgentLoopResult, or None
error = guard.trellar_evaluate_error # the exception, if the auto-triggered call failed
Requests are marked single_call: true in the payload. Not covered: agent.structured_output() and calling a Strands Model directly — Strands fires no model-call hooks for them. See orchestrations_examples/our_lab_with_aviran/single_llm_agent/simple_llm_call_strands.py.
Where to call evaluate_confidence
Call it from a graph node, at the point in the run you want scored, while the run is still in progress — the callback handler is released as soon as the root run ends, so calling it after invoke() returns raises ValueError. The payload is the events captured so far — later nodes are not included.
There are two ways to use the result:
1. Gate — validate before the graph continues
Put the call on an edge you do not want the graph to cross until Trellar has scored the run. Use result.score / result.explanation to decide whether to proceed or stop.
def confidence_gate(state):
result = evaluate_confidence()
if result.score < 7:
return {**state, "halt": True, "reason": result.explanation}
return {**state, "halt": False}
Wire that node in front of the next step, and only continue when the score is acceptable.
2. Observe — send a validation, do not restrict the graph
Put the call in any node where you want a score recorded. Store or log result if you want it; do not branch on it. The graph continues either way.
def report_confidence(state):
result = evaluate_confidence()
return {**state, "confidence_score": result.score, "confidence_explanation": result.explanation}
Environment Variables
The SDK always talks to the managed Trellar backend at https://api.trellar.io — this is fixed and cannot be overridden via an environment variable or function argument.
The API key itself is created only at trellar.io. Once you have it, you can pass it to evaluate_confidence(api_key=...) or set it as an environment variable so you do not pass it on every call:
| Variable | Description | Default |
|---|---|---|
TRELLAR_API_KEY |
Bearer token for authentication | (required) |
export TRELLAR_API_KEY=your-api-key
result = evaluate_confidence() # api_key read from the env var
API Reference
get_agent_guard
get_agent_guard(
agent_name: str,
observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler
| Parameter | Type | Description |
|---|---|---|
agent_name |
str |
Stable, unique name identifying this agent graph (e.g. "research-agent") |
observability_mode |
ObservabilityMode |
Controls whether evaluate_confidence() is auto-triggered when the graph run finishes. Default ObservabilityMode.NONE (no auto-trigger). |
Returns a LangChain callback handler bound to agent_name. Pass it to graph.invoke(..., config={"callbacks": [guard]}).
Raises:
ValueError— ifagent_nameis empty or blank
ObservabilityMode
Controls whether the guard automatically calls evaluate_confidence() for you when the graph run finishes (the root graph.invoke() call completes), so you don't have to add a manual call yourself.
| Value | Behavior |
|---|---|
ObservabilityMode.NONE |
Never auto-call. Default; identical to not passing observability_mode at all. |
ObservabilityMode.ALWAYS |
Always call evaluate_confidence() when the run finishes. |
ObservabilityMode.IF_NOT_EVALUATED |
Call evaluate_confidence() when the run finishes only if it was not already successfully called earlier in the run (e.g. from a gate node). |
from trellar import get_agent_guard, ObservabilityMode
guard = get_agent_guard("research-agent", ObservabilityMode.IF_NOT_EVALUATED)
graph.invoke(inputs, config={"callbacks": [guard]})
# evaluate_confidence() has already run automatically if no node called it.
Auto-triggered calls never raise: any error (missing API key, HTTP error, NetworkHaltedError, etc.) is caught and logged instead of propagating out of graph.invoke(). A manual call to evaluate_confidence() still raises normally.
Requests triggered this way are marked in the payload sent to the backend with observability_call: true (false for a normal, manually-invoked call), so the backend can distinguish automatic observability calls from explicit ones.
get_single_call_guard
get_single_call_guard(
agent_name: str,
observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler
Use this instead of get_agent_guard when you are calling a chat model directly (llm.invoke(...)) with no wrapping LangGraph/chain. Agents built with create_react_agent are already compiled graphs under the hood, so they work with get_agent_guard as usual — get_single_call_guard is only for a genuinely bare model call.
A bare llm.invoke() call has no node to call evaluate_confidence() from mid-run, and the callback handler is released as soon as the call finishes — so a manual call is never supported here. Use ObservabilityMode.ALWAYS (or IF_NOT_EVALUATED) to auto-trigger the evaluation, then read the result back from the guard:
from trellar import get_single_call_guard, ObservabilityMode
guard = get_single_call_guard("single-llm-call", ObservabilityMode.ALWAYS)
llm.invoke(messages, config={"callbacks": [guard]})
result = guard.trellar_evaluate_result # AgentLoopResult, or None if not yet evaluated
error = guard.trellar_evaluate_error # the exception, if the auto-triggered call failed
Requests made through this guard are marked in the payload with single_call: true (false for get_agent_guard), so the backend can tell the two apart.
evaluate_confidence
evaluate_confidence(
*,
api_key: str | None = None,
timeout: float = 30.0,
) -> AgentLoopResult
| Parameter | Type | Description |
|---|---|---|
api_key |
str | None |
Bearer token. Falls back to TRELLAR_API_KEY |
timeout |
float |
HTTP request timeout in seconds (default 30.0) |
context, trace_id, and agent_name are resolved automatically from the active guard created by get_agent_guard — there is no way to pass them manually. Requests always go to https://api.trellar.io; callers cannot redirect them.
Raises:
ValueError— if no active guard is found, itstrace_idcannot be resolved, orapi_keyis missingrequests.HTTPError— on non-2xx HTTP responses
AgentLoopResult
A frozen dataclass with:
| Field | Type | Description |
|---|---|---|
score |
int |
Confidence score from 1 (low) to 10 (high) |
explanation |
str |
Human-readable explanation of the score |
decision_identifier |
str |
Unique ID for this evaluation, for cross-referencing with the backend |
should_stop_network |
bool |
Whether the backend signaled the run should halt (see NetworkHaltedError) |
License
MIT — see LICENSE for details.
Metadata
Release files for trellar 0.6.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| trellar-0.6.1.tar.gz | 52.2 kB | With provenance | Details for trellar-0.6.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| trellar-0.6.1-py3-none-any.whl | Python 3 | none | any | Details for trellar-0.6.1-py3-none-any.whl |
Total release size: 83.1 kB
Release files / trellar-0.6.1.tar.gz
| Download URL | trellar-0.6.1.tar.gz |
|---|---|
| Size | 52.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9f53939c49973650dad3fc0bc934ccbe04525602fed73ae85610970ae2332973
|
|
BLAKE2b-256 checksum How to use checksums |
8d1c3c401e0411f6220e5550b35b678ec33825c29fa1f3aec310d6a9743fdbc0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 5, 2026.
Transparency logRelease files / trellar-0.6.1-py3-none-any.whl
| Download URL | trellar-0.6.1-py3-none-any.whl |
|---|---|
| Size | 30.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5786c9459fbd905036c134fc7fa63e4db16929e08ddfaa1f0ec8e208c2978281
|
|
BLAKE2b-256 checksum How to use checksums |
d3e6912a325170586462a9a5d90b7627882f1f78ff78be27a7449d63111ac8ff
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 5, 2026.
Transparency logSecurity for trellar 0.6.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for trellar-0.6.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 5, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.