tobeverified 0.5.1

Runtime verification for non-deterministic AI agent actions and outputs — Python SDK

tobeverified — Python SDK

Runtime verification for non-deterministic AI agent actions and outputs.

tobeverified submits each call to an independent verification agent (LLM judge with your policy + authoritative typologies, low-confidence cases route to a human reviewer). Use it where unit tests don't reach: refund classifications, fraud screens, PII redaction, intent routing, tool-call validation.

Install

pip install tobeverified

Use

from tobeverified import TBV

tbv = TBV()  # reads TBV_API_KEY and TBV_BASE_URL from env

result = tbv.verify(
    prompt="Is this customer message a refund request?",
    context={"message": "I'd like my money back."},
    allowed_verdicts=["yes", "no"],
    confidence_threshold=0.85,
)

print(result["final_verdict"], result["final_tier"])
# → 'yes' 'agent'   (or 'human' if confidence was below threshold)

verify() is fire-and-wait: submits, polls until terminal state. For non-blocking flows use submit() + wait() separately.

Decorator pattern

from tobeverified import verified

@verified(
    allowed_verdicts=["safe", "unsafe", "needs_review"],
    confidence_threshold=0.9,
    source="my-app/comment-mod",
)
def moderate(text: str) -> dict:
    return {
        "prompt": "Is this comment safe to publish?",
        "context": {"text": text},
    }

result = moderate("user-supplied content")
# result.verdict, result.tier, result.output

Get an API key

Sign in at app.tobeverified.com → API keys → create. Plaintext is shown exactly once.

export TBV_API_KEY=tbv_live_...

Reviewer queue

Verifications routed to human_needed (confidence below threshold) appear at review.tobeverified.com for the account that submitted them. Anyone signed in to the account can resolve. The verification agent's first-pass verdict, confidence, and rationale are shown alongside the verdict form.

Reference

Method	Purpose
TBV(base_url=None, api_key=None, timeout=30.0)	Construct a client. Defaults read TBV_BASE_URL and TBV_API_KEY from env.
submit(prompt, allowed_verdicts, ...)	Enqueue a verification task; returns {id, status, ...}.
get(task_id)	Fetch current state.
wait(task_id, timeout=300.0, poll_interval=1.5)	Block until terminal state.
verify(prompt, allowed_verdicts, ...)	submit() + wait() in one call.

Errors raise TBVError (or TBVTimeoutError for wait() timeouts). Both inherit from Exception.

Status

Developer preview. Versioned 0.x until the API contract is frozen; expect minor surface changes. Production deployments tracked via docs.tobeverified.com.