virgo-observe

Four Python operations for Virgo: register a client, trace an operation, attach feedback, and publish an individual product metric. Traces use OpenTelemetry; feedback and metrics use unsampled HTTP channels authorized by the same Observe key.

Version 0.5.1 adds certified fast paths for the nine most popular missing Python integrations. Python 3.10+ is supported. Install the published release and the single extra matching the application's highest-level framework:

pip install virgo-observe==0.5.1
pip install 'virgo-observe[litellm]==0.5.1'

For local development, build/install the checkout instead:

uv build packages/virgo-observe-python
pip install dist/virgo_observe-0.5.1-py3-none-any.whl

Four operations

Choose coverage in Virgo's “Trace with Virgo-Observe” setup and create one key. Configure it outside application code:

export VIRGO_API_KEY='<observe-key>'

The SDK defaults to hosted Virgo. For local/custom Virgo deployments, the generated UI/MCP setup prompt supplies the non-secret register(endpoint=...) address automatically. The copy block still contains only the API key. VIRGO_ENDPOINT remains available for advanced manual configuration.

from decimal import Decimal
from virgo_observe import MetricSubject, VirgoSubject, VirgoVersions, register

virgo = register(project_name="support-agent", product=True, metrics=True, release="release-7")

with virgo.agent_run(
    "support.resolve_ticket",
    run_ref="run-opaque-123",
    subject=VirgoSubject(account_ref="account-opaque-123"),
    versions=VirgoVersions(agent="agent-v4", prompt="prompt-v7"),
):
    # Run your agent here. Selected supported integrations emit child spans.
    with virgo.span("retrieval.context_pack", kind="RETRIEVER"):
        pass

    saved_execution = virgo.current_trace_ref()
    completed = virgo.metric(
        "resolved_ticket_value",
        observation_id="ticket-123-value",
        value=Decimal("12.34"),
        unit="USD",
        definition_version="resolved-ticket-v1",
        subject=MetricSubject("account", "account-opaque-123"),
        lineage={"producer": "ticket-service"},
    )

# Feedback may arrive later. Persist saved_execution with your own run record
# when the response/callback crosses a process boundary.
feedback = virgo.feedback(
    feedback_id="ticket-123-rating",
    trace=saved_execution,
    rating="down",
    kind="correction",
    correction="The answer should use the selected workspace.",
)

report = virgo.flush_report(timeout_seconds=5)
# Inspect report.feedback, report.metrics, and each receipt.status.
virgo.shutdown(timeout_seconds=5)

A saved TraceRef contains an external nonzero 32-character lowercase W3C trace ID and/or your opaque run reference, never a Virgo-internal ptr_… ID. An explicit reference wins over the active execution. Feedback requires one; metrics may remain deliberately unlinked. Subject references are opaque application IDs, not pre-hashed Virgo pseudonyms.

span() returns a native OpenTelemetry Span. kind is an optional AI role such as AGENT, LLM, or TOOL; otel_kind=SpanKind.CLIENT independently preserves native transport semantics. Generic spans are not inferred to be tools from parentage. agent_run(), get_tracer(), flush(), and instrumented_frameworks remain supported. Typed identity, run, release, and version arguments own their reserved attribute keys.

Point observations and real outcome windows

A metric without window is one observed event: it requires a non-null boolean/numeric value, revision 1, and no supersession. Its occurrence is captured once when called, or set explicitly with an aware occurred_at. It does not fabricate a time window or interpret False as a mature failure. Use an event KPI definition to aggregate these observations; precomputed custom KPI definitions consume windowed measurements only.

For a completed or censored outcome window, supply MetricWindow(start, end, mature_at) using timezone-aware datetimes. The order is start < end <= mature_at. An observed window must already be mature; occurred_at, if supplied, must equal its end. Supported statuses are observed, immature, proxy, right_censored, not_achieved, and reversed. Immature/right-censored values are null.

value accepts bool, int, finite float, finite bounded Decimal, or None where the status permits it. Decimal is serialized as exact decimal text, not a float. Arbitrary categorical strings are rejected. Subjects have exactly one grain: account, user, journey, conversation, session, or aggregate (without a ref). Approved lineage keys are source, source_version, definition_hash, export_id, and producer.

Corrections retain the logical observation_id, increment revision, and set supersedes_observation_id. Every revision gets a distinct stable HTTP idempotency key. Retrying an unchanged revision is a duplicate; changing its semantic value, timestamp, lineage, subject, release, versions, or execution claim is a conflict. New revisions never edit the previous record.

Privacy and integration ownership

Full content is the default: prompts, responses, tool inputs/outputs, and retrieval content. Observe onboarding also configures the server source for full content. Explicit capture_content=False remains available when an application intentionally needs metadata-only export. When the argument is omitted, VIRGO_CAPTURE_CONTENT=true|false can override the default. Explicit arguments win; the server's trace-source policy remains authoritative.

In explicit metadata-only mode, the exporter filters its own representation: unapproved attributes, events (including exception content), status descriptions, link attributes, and resource/scope attributes are removed. It does not mutate spans delivered to another exporter. Operational names and approved opaque IDs remain; do not put prompts, personal data, secrets, or arbitrary content in those fields. This trace setting does not erase explicitly submitted feedback text, which follows the feedback source's policy (full content for Observe onboarding).

Explicit metadata-only export retains vcs.ref.head.revision and platform.commit.sha only as strings of exactly 40 hexadecimal characters. Set the emitter commit on the OpenTelemetry Resource and provide platform.release.id (for example, via release=) for canonical trace source provenance. Span-only commit attributes do not establish that provenance. Other source-control fields and arbitrary evaluator or deployment metadata remain filtered. This does not reconstruct historical provenance.

product=True and metrics=True independently enable publication using the same key. Their default URLs use the trace endpoint's origin; the backend resolves the workspace from the key. Disabled channels are not configured. Ordinary legacy trace keys do not gain publication access: enable coverage through Observe setup. Rotating or revoking its trace-source key affects all selected channels. Existing explicit feedback/outcome tokens remain supported for older integrations; a shared key is never forwarded to a different origin.

Register before constructing instrumented clients. Install only the one extra matching the application's highest-level framework, then select its explicit name. Each extra pins Virgo's adapter. Some upstream adapters declare a minimum framework version, so resolve the one chosen extra through the application's existing lock and never force an unrelated framework upgrade.

Semantic Kernel 1.44.1 uses its native OpenTelemetry spans, so it needs no adapter extra. Set SEMANTICKERNEL_EXPERIMENTAL_GENAI_ENABLE_OTEL_DIAGNOSTICS_SENSITIVE=true before importing Semantic Kernel, install base virgo-observe==0.5.1, and use instrumentors=["semantic_kernel"]. Semantic Kernel's native tracer resolves through the global SDK TracerProvider; pass that same provider explicitly to register(). Virgo bridges the opted-in prompt/response log events onto the native model span so its trace exporter applies the same content policy.

Installing an arbitrary OpenInference entry point is not a support guarantee. Use auto_instrument=False for manual tracing.

Automatic selection activates framework, then gateway, then provider layers. An active higher-level layer prevents duplicate lower-level model spans. Overlapping gateway or provider adapters are reported as unsupported with overlapping_model_instrumentation; direct provider calls outside the selected higher-level framework then need a provider-only selection or manual spans. Existing external instrumentation is never reconfigured and is reported already_active_unverified, not healthy by inference.

Inspect virgo.registration_report for each integration's distribution version, status, ownership, and safe reason code. Discovery/activation is not proof of server delivery. Native automatic metadata-only instrumentation requires patched Pydantic AI 2.27.1+; earlier versions are skipped because retry content can leak through their native instrumentation.

The reproducible integration matrix uses real SDKs and local fake responses, including model streaming and native tool/validator retries:

Other versions are not individually certified. Install optional integrations alongside the application's existing dependencies; do not upgrade an application merely to silence an unsupported registration report.

Providers, configuration, and lifetime

register() uses an explicitly supplied SDK provider, then an existing global SDK provider, or otherwise a private provider. It never replaces the global provider. Compatible registrations share one export stream and sender with reference-counted lifetime; conflicting registrations on that provider fail. Closing one handle does not shut down another handle or the customer's provider. Final shutdown releases Virgo-owned framework wrappers only while their ownership still matches; externally installed or replaced wrappers are left untouched.

In pre-fork servers, create the provider and call register() inside each worker after it starts. Do not share a live Virgo handle or delivery receipt across a fork. Inherited handles reject operations instead of using stale threads/locks. Use explicit shutdown for short-lived jobs; process exit is not a delivery receipt.

trace_export="existing" requires an existing SDK provider, needs no Virgo trace key, and attaches no Virgo exporter. The application's exporter and privacy policy own that path; Virgo's export-local content filter does not apply to it. The independent feedback/metric channels still work.

Explicit arguments take precedence over the channel environment variables. Trace compatibility also accepts OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_TRACES_HEADERS/OTEL_EXPORTER_OTLP_HEADERS, and OTEL_SERVICE_NAME. VIRGO_ENDPOINT, VIRGO_ENVIRONMENT, and VIRGO_RELEASE override hosted defaults. The project display name is never used to guess a workspace publication URL. Missing channels are permitted at registration, but publishing to one raises a configuration error.

Each record channel has its own finite in-memory queue and worker (default 512 pending events each). Payloads, occurrence times, IDs, and operational W3C headers are captured before enqueueing. Network calls suppress recursive HTTP instrumentation. At most three attempts retry transport errors, 408, 429, and 5xx responses with bounded jitter/backoff and Retry-After. Other HTTP failures are terminal; redirects never forward credentials.

Receipts expose queued, accepted, rejected, conflict, failed, or dropped states. Queue overflow is visible as dropped, not silently successful. A 202 means accepted at ingress, not normalized, linked, or successfully analyzed. flush_report() reports cumulative failures and pending records; flush() is its boolean compatibility view. Trace force-flush completion is not proof of remote ingestion.

Flush and shutdown share a total caller deadline across channels. An exporter or in-flight HTTP call can finish after the caller's deadline; Python cannot safely cancel arbitrary third-party I/O. Shutdown marks undelivered receipts failed and closes owned resources. This queue is not crash-durable: persist important application events in your own outbox and reuse their IDs when retrying after a process restart. Call shutdown during orderly teardown.

Server contract and verification

Point metrics, explicit observation/release/version fields, and durable privacy-aware correlations require the accompanying Platform server changes and forward migration 0197. Upgrade via platform-migrate; never use an SDK client-side fallback against an older window-only endpoint.

Exact links require the same tenant, workspace, environment, and permitted pseudonym scope. Multiple candidates and contradictory references abstain. Late links live in a separate projection, not edits to immutable outcome rows. Pending claims receive bounded retries and a final expiry check; missing or sampled-out traces do not prevent observation acceptance.

From the Platform repository root:

uv run ruff format packages/virgo-observe-python
uv run ruff check packages/virgo-observe-python
uv run mypy packages/virgo-observe-python/src
uv run pytest packages/virgo-observe-python/tests
uv run python packages/virgo-observe-python/scripts/check_framework_matrix.py
uv build packages/virgo-observe-python
uv run python scripts/verify_virgo_observe_distribution.py

See docs/virgo/instrumentation/python.md, docs/observability.md, and docs/virgo/instrumentation/python-release.md for server inspection and the separately authorized publication procedure.

License

The virgo-observe Python SDK is licensed under the Apache License, Version 2.0. The full license is included in this package's LICENSE file and its wheel and source distributions.

This license applies only to packages/virgo-observe-python within the Platform repository. It does not license the rest of the Platform repository or Virgo's hosted services. Third-party dependencies retain their own licenses.