Skip to main content

trellar

PyPI version Python License: MIT CI

A lightweight Python client for the Trellar confidence evaluation API. Attach a callback to your LangChain / LangGraph run, then call evaluate_confidence() when you want a score. Context, trace ID, and agent name are picked up automatically — no manual wiring.

This library cannot be used without an API key from trellar.io.


Create an account and API key

trellar.io is the only place that issues API keys for this library. Create an account there, then generate an API key from the dashboard. Without that key, evaluate_confidence() cannot authenticate and the client will not work.

Then pass the key into the SDK (see Environment Variables):

  • evaluate_confidence(api_key="..."), or
  • TRELLAR_API_KEY in the environment

Installation

pip install "trellar[langchain]"

The langchain extra is required because agent runs are captured via a LangChain callback handler (get_agent_guard). Requires Python 3.9+.


Quick Start

from trellar import get_agent_guard, evaluate_confidence

# agent_name must be a stable, unique name for this agent graph — the
# backend uses it to track the graph's network profile across runs.
guard = get_agent_guard("research-agent")

# call evaluate_confidence() from a node, while the run is still in progress —
# context, trace_id, and agent_name are picked up from the guard automatically
def report_confidence(state):
    result = evaluate_confidence()
    print(result.score)        # int, 1-10
    print(result.explanation)  # str, human-readable reasoning
    return state

graph.invoke(inputs, config={"callbacks": [guard]})

Where to call evaluate_confidence

Call it from a graph node, at the point in the run you want scored, while the run is still in progress — the callback handler is released as soon as the root run ends, so calling it after invoke() returns raises ValueError. The payload is the events captured so far — later nodes are not included.

There are two ways to use the result:

1. Gate — validate before the graph continues

Put the call on an edge you do not want the graph to cross until Trellar has scored the run. Use result.score / result.explanation to decide whether to proceed or stop.

def confidence_gate(state):
    result = evaluate_confidence()
    if result.score < 7:
        return {**state, "halt": True, "reason": result.explanation}
    return {**state, "halt": False}

Wire that node in front of the next step, and only continue when the score is acceptable.

2. Observe — send a validation, do not restrict the graph

Put the call in any node where you want a score recorded. Store or log result if you want it; do not branch on it. The graph continues either way.

def report_confidence(state):
    result = evaluate_confidence()
    return {**state, "confidence_score": result.score, "confidence_explanation": result.explanation}

Environment Variables

The SDK always talks to the managed Trellar backend at https://api.trellar.io — this is fixed and cannot be overridden via an environment variable or function argument.

The API key itself is created only at trellar.io. Once you have it, you can pass it to evaluate_confidence(api_key=...) or set it as an environment variable so you do not pass it on every call:

Variable Description Default
TRELLAR_API_KEY Bearer token for authentication (required)
export TRELLAR_API_KEY=your-api-key
result = evaluate_confidence()  # api_key read from the env var

API Reference

get_agent_guard

get_agent_guard(
    agent_name: str,
    observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler
Parameter Type Description
agent_name str Stable, unique name identifying this agent graph (e.g. "research-agent")
observability_mode ObservabilityMode Controls whether evaluate_confidence() is auto-triggered when the graph run finishes. Default ObservabilityMode.NONE (no auto-trigger).

Returns a LangChain callback handler bound to agent_name. Pass it to graph.invoke(..., config={"callbacks": [guard]}).

Raises:

  • ValueError — if agent_name is empty or blank

ObservabilityMode

Controls whether the guard automatically calls evaluate_confidence() for you when the graph run finishes (the root graph.invoke() call completes), so you don't have to add a manual call yourself.

Value Behavior
ObservabilityMode.NONE Never auto-call. Default; identical to not passing observability_mode at all.
ObservabilityMode.ALWAYS Always call evaluate_confidence() when the run finishes.
ObservabilityMode.IF_NOT_EVALUATED Call evaluate_confidence() when the run finishes only if it was not already successfully called earlier in the run (e.g. from a gate node).
from trellar import get_agent_guard, ObservabilityMode

guard = get_agent_guard("research-agent", ObservabilityMode.IF_NOT_EVALUATED)
graph.invoke(inputs, config={"callbacks": [guard]})
# evaluate_confidence() has already run automatically if no node called it.

Auto-triggered calls never raise: any error (missing API key, HTTP error, NetworkHaltedError, etc.) is caught and logged instead of propagating out of graph.invoke(). A manual call to evaluate_confidence() still raises normally.

Requests triggered this way are marked in the payload sent to the backend with observability_call: true (false for a normal, manually-invoked call), so the backend can distinguish automatic observability calls from explicit ones.

get_single_call_guard

get_single_call_guard(
    agent_name: str,
    observability_mode: ObservabilityMode = ObservabilityMode.NONE,
) -> BaseCallbackHandler

Use this instead of get_agent_guard when you are calling a chat model directly (llm.invoke(...)) with no wrapping LangGraph/chain. Agents built with create_react_agent are already compiled graphs under the hood, so they work with get_agent_guard as usual — get_single_call_guard is only for a genuinely bare model call.

A bare llm.invoke() call has no node to call evaluate_confidence() from mid-run, and the callback handler is released as soon as the call finishes — so a manual call is never supported here. Use ObservabilityMode.ALWAYS (or IF_NOT_EVALUATED) to auto-trigger the evaluation, then read the result back from the guard:

from trellar import get_single_call_guard, ObservabilityMode

guard = get_single_call_guard("single-llm-call", ObservabilityMode.ALWAYS)
llm.invoke(messages, config={"callbacks": [guard]})

result = guard.trellar_evaluate_result   # AgentLoopResult, or None if not yet evaluated
error = guard.trellar_evaluate_error     # the exception, if the auto-triggered call failed

Requests made through this guard are marked in the payload with single_call: true (false for get_agent_guard), so the backend can tell the two apart.

evaluate_confidence

evaluate_confidence(
    *,
    api_key: str | None = None,
    timeout: float = 30.0,
) -> AgentLoopResult
Parameter Type Description
api_key str | None Bearer token. Falls back to TRELLAR_API_KEY
timeout float HTTP request timeout in seconds (default 30.0)

context, trace_id, and agent_name are resolved automatically from the active guard created by get_agent_guard — there is no way to pass them manually. Requests always go to https://api.trellar.io; callers cannot redirect them.

Raises:

  • ValueError — if no active guard is found, its trace_id cannot be resolved, or api_key is missing
  • requests.HTTPError — on non-2xx HTTP responses

AgentLoopResult

A frozen dataclass with:

Field Type Description
score int Confidence score from 1 (low) to 10 (high)
explanation str Human-readable explanation of the score
decision_identifier str Unique ID for this evaluation, for cross-referencing with the backend
should_stop_network bool Whether the backend signaled the run should halt (see NetworkHaltedError)

License

MIT — see LICENSE for details.

Metadata

Release files for trellar 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for trellar 0.5.0
File Size Uploaded Provenance Actions
trellar-0.5.0.tar.gz 40.0 kB With provenance Details for trellar-0.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for trellar 0.5.0
File Interpreter ABI Platform Actions
trellar-0.5.0-py3-none-any.whl Python 3 none any Details for trellar-0.5.0-py3-none-any.whl

Total release size: 60.8 kB

Release files / trellar-0.5.0.tar.gz

Download URL trellar-0.5.0.tar.gz
Size 40.0 kB
Tags Source
SHA-256 checksum
How to use checksums
b153c038ac0201ae9b6fb7e7eeec42d4a01ddefe4502bebdb83f2eb013ad9eed
BLAKE2b-256 checksum
How to use checksums
540d8bd3fbd8118359251f3d875a23155092578d6dfb6a9264461c3816fbf294
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 21, 2026.

Transparency log

Release files / trellar-0.5.0-py3-none-any.whl

Download URL trellar-0.5.0-py3-none-any.whl
Size 20.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8eaddc45e251cd72cf768ea232ca3ea8fab64435089dbfcf586bf3caf4ce4a7c
BLAKE2b-256 checksum
How to use checksums
b4059bd18d1810343faff48c56d8c34383a6b47a220a0e90552d6710e01aebcd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 21, 2026.

Transparency log

Security for trellar 0.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for trellar-0.5.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 21, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page