Skip to main content

traceAI-typesafe

OpenTelemetry instrumentation for the TypeSafe AI Python SDK (typesafe-sdk), the client for Jev's System One decision API (POST /v1/systemone).

Every TypeSafeClient.system_one(...) and AsyncTypeSafeClient.system_one(...) call becomes one EVALUATOR span in Future AGI, however many questions it asks. Each question becomes typesafe.question.<name>.* attributes:

Answer type Attributes
noul type, noul (probability of yes, a number, never Pass/Fail)
choice type, choice (the label), confidence
score type, score (expected score, a number), confidence
box type, box.top, box.bottom, box.left, box.right, confidence when not null

The span also carries gen_ai.provider.name=typesafe, gen_ai.request.model (as you asked for it), gen_ai.response.model plus typesafe.response.model when the API resolved an alias, gen_ai.usage.input_tokens / output_tokens, typesafe.request_id, server.address and typesafe.question_count. Probability maps, score legends and image bytes are never recorded. The API returns no explanation, so the span has none.

This package traces calls your code makes to TypeSafe. It does not add Jev as a judge model for Future AGI evals, and it does not call the API itself.

Install

pip install traceAI-typesafe typesafe-sdk

Use

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_typesafe import TypeSafeInstrumentor
from typesafe_sdk import Choice, Noul, TypeSafeClient

trace_provider = register(project_type=ProjectType.OBSERVE, project_name="support-triage")
TypeSafeInstrumentor().instrument(tracer_provider=trace_provider)

with TypeSafeClient() as client:  # reads TYPESAFE_API_KEY
    result = client.system_one(
        state="I was charged twice. Please help.",
        questions={
            "billing": Noul(instructions="Is this about billing?"),
            "tone": Choice(instructions="What is the tone?", criteria={"calm": None, "angry": None}),
        },
    )
trace_provider.force_flush()

The instrumentor never sets base_url or a model. Point the client wherever you already do (base_url= or TYPESAFE_BASE_URL); server.address records the host.

Privacy

Like every traceAI instrumentor, content is captured by default:

Setting Effect
default input.value = the state and the questions (instructions, criteria); output.value and typesafe.question.<name>.choice = the decisions
FI_HIDE_INPUTS=true (or TraceConfig(hide_inputs=True), FI_HIDE_INPUT_TEXT) no input.value, no typesafe.idempotency_key
FI_HIDE_OUTPUTS=true (or TraceConfig(hide_outputs=True), FI_HIDE_OUTPUT_TEXT) no output.value, no choice labels; numeric answers stay
FI_PII_REDACTION=true regex PII redaction on every string attribute and the error message

Always recorded: question names and types (they are attribute keys, so do not put secrets in question names), numeric answers, model, usage, request id. Never recorded: the API key, stack traces, the text of client-side errors (they can name a local file), image bytes or paths. The API's own error message is added to the span status only while neither inputs nor outputs are hidden.

Errors and retries

An HTTP error (401, 403, 422, 429, 5xx) or a connection error ends the span with ERROR, error.type, http.response.status_code, the server's typesafe.error_type and the request id. The instrumentor adds no retries. The SDK's own RetryPolicy (by default up to 2 retries for 408, 429 and 5xx) runs inside the one span, so a 429 followed by a success is one OK span. A call you repeat yourself is a second span.

Tests

pip install -e . pytest pytest-asyncio
pytest

The tests run the real typesafe-sdk clients against an in-process fake shaped like TypeSafe's published OpenAPI document. They never contact api.typesafe.ai.

Metadata

Release files for traceAI-typesafe 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for traceAI-typesafe 0.1.0
File Size Uploaded Provenance Actions
traceai_typesafe-0.1.0.tar.gz 12.2 kB Without provenance Details for traceai_typesafe-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for traceAI-typesafe 0.1.0
File Interpreter ABI Platform Actions
traceai_typesafe-0.1.0-py3-none-any.whl Python 3 none any Details for traceai_typesafe-0.1.0-py3-none-any.whl

Total release size: 24.6 kB

Release files / traceai_typesafe-0.1.0.tar.gz

Download URL traceai_typesafe-0.1.0.tar.gz
Size 12.2 kB
Tags Source
SHA-256 checksum
How to use checksums
3ac1e52bd471881fdbd059fd8677459beb04415aae0ca9884186182e94880223
BLAKE2b-256 checksum
How to use checksums
bf1976f42f5f04c62cabdafdc168de135b1101a69884bccca8d5b61ec6bddfe6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / traceai_typesafe-0.1.0-py3-none-any.whl

Download URL traceai_typesafe-0.1.0-py3-none-any.whl
Size 12.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3749d0370f057a92e5d4030b33c47e5acb086b53feec26d4b193dc0aff39fc0e
BLAKE2b-256 checksum
How to use checksums
ee2ee32e34aa60d7c3cd856fa8480d7d81b5f8a2c1e7f5a02d233d9c2962207c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for traceAI-typesafe 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page