Skip to main content

trellar

PyPI version Python License: MIT CI

A lightweight Python client for the Trellar confidence evaluation API. Attach a callback to your LangChain / LangGraph run, then call evaluate_confidence() when you want a score. Context, trace ID, and agent name are picked up automatically — no manual wiring.

This library cannot be used without an API key from trellar.io.


Create an account and API key

trellar.io is the only place that issues API keys for this library. Create an account there, then generate an API key from the dashboard. Without that key, evaluate_confidence() cannot authenticate and the client will not work.

Then pass the key into the SDK (see Environment Variables):

  • evaluate_confidence(api_key="..."), or
  • TRELLAR_API_KEY in the environment

Installation

pip install "trellar[langchain]"

The langchain extra is required because agent runs are captured via a LangChain callback handler (get_agent_guard). Requires Python 3.9+.


Quick Start

from trellar import get_agent_guard, evaluate_confidence

# agent_name must be a stable, unique name for this agent graph — the
# backend uses it to track the graph's network profile across runs.
guard = get_agent_guard("research-agent")

# call evaluate_confidence() from a node, while the run is still in progress —
# context, trace_id, and agent_name are picked up from the guard automatically
def report_confidence(state):
    result = evaluate_confidence()
    print(result.score)        # int, 1-10
    print(result.explanation)  # str, human-readable reasoning
    return state

graph.invoke(inputs, config={"callbacks": [guard]})

Strands Agents

pip install "trellar[strands]"   # requires Python 3.10+
from strands import Agent
from strands.multiagent import GraphBuilder
from trellar import get_strands_guard, evaluate_confidence

guard = get_strands_guard("research-agent")

# Give every agent a stable name, and register the guard on each agent...
searcher = Agent(name="searcher", hooks=[guard])
reporter = Agent(name="reporter", hooks=[guard])

# ...and on the Graph/Swarm, so the whole run is one trace.
builder = GraphBuilder()
builder.add_node(searcher, "searcher")
builder.add_node(reporter, "reporter")
builder.add_edge("searcher", "reporter")
builder.set_entry_point("searcher")
builder.set_hook_providers([guard])
graph = builder.build()

graph("Find me something to report on")

Call evaluate_confidence() from inside the run (a graph node or a tool), exactly as with LangChain. ObservabilityMode works the same way. See orchestrations_examples/our_lab_with_aviran/car_stocks_buy_mcp_celery.py/car_stocks_buy_from_remote_provider_orchestration_strands.py for a full example.

get_strands_single_call_guard

For one Agent called once (no Graph/Swarm). The run is over when the call returns, so use ObservabilityMode.ALWAYS (or IF_NOT_EVALUATED) and read the result off the guard:

from strands import Agent
from trellar import get_strands_single_call_guard, ObservabilityMode

guard = get_strands_single_call_guard("faq-agent", ObservabilityMode.ALWAYS)
agent = Agent(name="faq_agent", hooks=[guard])
agent("What time does the office open?")

result = guard.trellar_evaluate_result   # AgentLoopResult, or None
error = guard.trellar_evaluate_error     # the exception, if the auto-triggered call failed

Requests are marked single_call: true in the payload. Not covered: agent.structured_output() and calling a Strands Model directly — Strands fires no model-call hooks for them. See orchestrations_examples/our_lab_with_aviran/single_llm_agent/simple_llm_call_strands.py.


Where to call evaluate_confidence

Call it from a graph node, at the point in the run you want scored, while the run is still in progress — the callback handler is released as soon as the root run ends, so calling it after invoke() returns raises ValueError. The payload is the events captured so far — later nodes are not included.

There are two ways to use the result:

1. Gate — validate before the graph continues

Put the call on an edge you do not want the graph to cross until Trellar has scored the run. Use result.score / result.explanation to decide whether to proceed or stop.

def confidence_gate(state):
    result = evaluate_confidence()
    if result.score < 7:
        return {**state, "halt": True, "reason": result.explanation}
    return {**state, "halt": False}

Wire that node in front of the next step, and only continue when the score is acceptable.

2. Observe — send a validation, do not restrict the graph

Put the call in any node where you want a score recorded. Store or log result if you want it; do not branch on it. The graph continues either way.

def report_confidence(state):
    result = evaluate_confidence()
    return {**state, "confidence_score": result.score, "confidence_explanation": result.explanation}

Environment Variables

The SDK always talks to the managed Trellar backend at https://api.trellar.io — this is fixed and cannot be overridden via an environment variable or function argument.

The API key itself is created only at trellar.io. Once you have it, you can pass it to evaluate_confidence(api_key=...) or set it as an environment variable so you do not pass it on every call:

Variable Description Default
TRELLAR_API_KEY Bearer token for authentication (required)
export TRELLAR_API_KEY=your-api-key
result = evaluate_confidence()  # api_key read from the env var

API Reference

get_agent_guard

get_agent_guard(
    agent_name: str,
    observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler
Parameter Type Description
agent_name str Stable, unique name identifying this agent graph (e.g. "research-agent")
observability_mode ObservabilityMode Controls whether evaluate_confidence() is auto-triggered when the graph run finishes. Default ObservabilityMode.NONE (no auto-trigger).

Returns a LangChain callback handler bound to agent_name. Pass it to graph.invoke(..., config={"callbacks": [guard]}).

Raises:

  • ValueError — if agent_name is empty or blank

ObservabilityMode

Controls whether the guard automatically calls evaluate_confidence() for you when the graph run finishes (the root graph.invoke() call completes), so you don't have to add a manual call yourself.

Value Behavior
ObservabilityMode.NONE Never auto-call. Default; identical to not passing observability_mode at all.
ObservabilityMode.ALWAYS Always call evaluate_confidence() when the run finishes.
ObservabilityMode.IF_NOT_EVALUATED Call evaluate_confidence() when the run finishes only if it was not already successfully called earlier in the run (e.g. from a gate node).
from trellar import get_agent_guard, ObservabilityMode

guard = get_agent_guard("research-agent", ObservabilityMode.IF_NOT_EVALUATED)
graph.invoke(inputs, config={"callbacks": [guard]})
# evaluate_confidence() has already run automatically if no node called it.

Auto-triggered calls never raise: any error (missing API key, HTTP error, NetworkHaltedError, etc.) is caught and logged instead of propagating out of graph.invoke(). A manual call to evaluate_confidence() still raises normally.

Requests triggered this way are marked in the payload sent to the backend with observability_call: true (false for a normal, manually-invoked call), so the backend can distinguish automatic observability calls from explicit ones.

get_single_call_guard

get_single_call_guard(
    agent_name: str,
    observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler

Use this instead of get_agent_guard when you are calling a chat model directly (llm.invoke(...)) with no wrapping LangGraph/chain. Agents built with create_react_agent are already compiled graphs under the hood, so they work with get_agent_guard as usual — get_single_call_guard is only for a genuinely bare model call.

A bare llm.invoke() call has no node to call evaluate_confidence() from mid-run, and the callback handler is released as soon as the call finishes — so a manual call is never supported here. Use ObservabilityMode.ALWAYS (or IF_NOT_EVALUATED) to auto-trigger the evaluation, then read the result back from the guard:

from trellar import get_single_call_guard, ObservabilityMode

guard = get_single_call_guard("single-llm-call", ObservabilityMode.ALWAYS)
llm.invoke(messages, config={"callbacks": [guard]})

result = guard.trellar_evaluate_result   # AgentLoopResult, or None if not yet evaluated
error = guard.trellar_evaluate_error     # the exception, if the auto-triggered call failed

Requests made through this guard are marked in the payload with single_call: true (false for get_agent_guard), so the backend can tell the two apart.

evaluate_confidence

evaluate_confidence(
    *,
    api_key: str | None = None,
    timeout: float = 30.0,
) -> AgentLoopResult
Parameter Type Description
api_key str | None Bearer token. Falls back to TRELLAR_API_KEY
timeout float HTTP request timeout in seconds (default 30.0)

context, trace_id, and agent_name are resolved automatically from the active guard created by get_agent_guard — there is no way to pass them manually. Requests always go to https://api.trellar.io; callers cannot redirect them.

Raises:

  • ValueError — if no active guard is found, its trace_id cannot be resolved, or api_key is missing
  • requests.HTTPError — on non-2xx HTTP responses

AgentLoopResult

A frozen dataclass with:

Field Type Description
score int Confidence score from 1 (low) to 10 (high)
explanation str Human-readable explanation of the score
decision_identifier str Unique ID for this evaluation, for cross-referencing with the backend
should_stop_network bool Whether the backend signaled the run should halt (see NetworkHaltedError)

License

MIT — see LICENSE for details.

Metadata

Release files for trellar 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for trellar 0.6.0
File Size Uploaded Provenance Actions
trellar-0.6.0.tar.gz 50.6 kB With provenance Details for trellar-0.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for trellar 0.6.0
File Interpreter ABI Platform Actions
trellar-0.6.0-py3-none-any.whl Python 3 none any Details for trellar-0.6.0-py3-none-any.whl

Total release size: 80.4 kB

Release files / trellar-0.6.0.tar.gz

Download URL trellar-0.6.0.tar.gz
Size 50.6 kB
Tags Source
SHA-256 checksum
How to use checksums
cbf3afbfafcda7a0c454c2e656c8f09c67be3c9ba093cd08024d42d546a31c3b
BLAKE2b-256 checksum
How to use checksums
0df8f54e4a1b488f831a2ef0827cce8a8bb280200f3b22278b229902b7daf687
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 3, 2026.

Transparency log

Release files / trellar-0.6.0-py3-none-any.whl

Download URL trellar-0.6.0-py3-none-any.whl
Size 29.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0578544e8453980d82397726ff3737ea8f6d1837297384155598756c69f44ad9
BLAKE2b-256 checksum
How to use checksums
db73ee1420371a95fc7644f3008aa0cdc5fdbff8532847b0235057e43fbaedb5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 3, 2026.

Transparency log

Security for trellar 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for trellar-0.6.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 3, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page