finest-ai 0.1.0

Finest observer SDK — see every job your code asks an AI model to do, without changing what it does.

finest — the Python observer SDK

See every job your code asks an AI model to do. Change nothing about how it does it.

import finest_ai
from openai import OpenAI

finest.init(
    api_key="fnst_...",
    environment="production",
    spool_dir="/var/lib/my-app/finest-spool",
)
client = finest.wrap(OpenAI())

client.chat.completions.create(
    model="...",
    messages=[...],
    finest={"route": "classify_support_ticket"},   # stripped before the request is sent
)

Two calls. Your requests still go from your process straight to your provider, on your key. The in-process observer inspects the call to derive metadata; normal request and response payloads do not transit or persist in Finest's control plane.

The Finest key authenticates a startup bootstrap against https://api.finest.so: workspace identity, policy verification keys, revocation epoch, and hard constraints become visible in diagnostics(). This Python release remains deliberately observer-only even when bootstrap publishes routing endpoints; it never presents bootstrap readiness as routing support.

For a local observer that must make no Finest network calls:

finest.init(offline=True)

What it does

It records metadata about each model call — which job, which configuration, how many tokens, how long, what the provider said it actually served — and hands that to the control plane. That is what lets the console say "your app asks AI to do four jobs, and job 1 costs $84/month" before anything is changed.

Content is not captured. Prompts and completions never leave your process.

What it does not do

It does not route. Nothing this SDK observes can be sent to a different model: there is no policy evaluation and no adapter in this tier. Routing is a separate capability gated on a signed policy, and when it arrives in Python it will arrive behind finest.policy, which already verifies exactly the same signatures the TypeScript SDK does — see Cross-language parity.

The three promises

It never raises into your call. Every stage around the provider call — detection, annotation, hashing, journalling — runs inside a guard that logs once and continues. tests/test_never_raises.py injects a failure at each stage in turn and asserts your result comes back untouched. The provider call itself is never wrapped: if OpenAI raises, you get that exception with its own traceback.

Its healthy path does not block your call. Recording is a bounded-queue put; spooling and upload happen on a daemon thread, batched, with an atexit flush. If that queue is already full, Finest fsyncs a local gap marker before returning so overload cannot become a silent hole in savings evidence.

FINEST_DISABLE=1 removes it entirely. wrap() returns the object you passed in — the same object, asserted by identity in the test suite. Not a disabled proxy: no proxy at all.

Install

Python ≥ 3.10. One runtime dependency, httpx, imported lazily inside the journal upload — a process without it still wraps, annotates and spools to disk; only the upload stops, and that shows up as a gap marker rather than an error.

The package is not published, and the PyPI name finest belongs to the unrelated FineST spatial-transcriptomics project. Do not run pip install finest. Build and install the explicit wheel from this repository:

python -m pip install hatchling build
python -m build packages/sdk-python
pip install ./packages/sdk-python/dist/*.whl

Route annotations

A route name describes the job, not the model:

finest={"route": "extract_invoice_fields", "sticky_key": conversation_id}

Good: classify_support_ticket, summarise_call_transcript, extract_invoice_fields. Bad: gpt_call, claude_route_2. The model behind a route is expected to change — that is the entire point — so a model name in a route name is a name that will be wrong next month.

sticky_key is what keeps a conversation, document or user on one side of a canary. Sticky assignment is never by request id.

The raw value never enters the journal. Hosted bootstrap supplies a stable workspace HMAC key; offline installs may set FINEST_STICKY_HMAC_KEY or pass sticky_hmac_key= (minimum 32 bytes). The SDK writes only a purpose-separated stk_v1_journal_… digest. Without a valid key it writes null, never the raw identifier.

Calls with no annotation are still observed: they are grouped by template hash under an unannotated: key and marked observer-only. An unannotated call is never routed, by construction.

What lands in the journal

The wire format is ingestJournalRequestSchema from @finest-ai/schemas — the same contract the TypeScript SDK writes:

{
  "sdkVersion": "finest-python/0.1.0",
  "attempts": [{ "attemptId": "01J...", "routeId": "rt_01J...", "usage": {...}, "...": "..." }],
  "gapMarkers": [{ "fromTs": 0, "toTs": 0, "reason": "sink_unreachable", "estimatedLost": 3 }]
}

Fields an in-process observer genuinely cannot determine — the serving region, the pinned model version when the provider does not report one — are the literal string "unreported". A plausible-looking guess in a field the ledger reconciles against would be worse than an absence.

Gap markers

Anything the journal loses becomes a gap marker: an uploaded record saying "attempts existed here and we do not have them". Reasons are spool_overflow, process_exit, sink_unreachable, disk_full, unknown.

A gap freezes savings evidence for its interval. That is deliberate: a period we cannot account for cannot support a trustworthy savings claim. Usage-based fee statement authority is not implemented in this release.

Two consequences of the ingest contract, both deliberate:

• A gap marker can travel alone. A gap-only batch is uploaded immediately; it does not wait for a later provider call to make a known evidence hole visible. Stable gap IDs make retries idempotent.
• A spooled-but-unsent attempt is not a gap. It is delayed, and the next process delivers it. Claiming a gap there would tell you your telemetry has a hole when it does not.

Durability

Attempts are appended to an NDJSON spool and fsync-ed before the network is attempted, so a kill -9 between spool and upload loses nothing — the next process picks the spool up. Attempt ids are ULIDs and ingest dedupes on them, so an interrupted drain re-sends rather than duplicates. A torn trailing line is dropped and reported as a process_exit gap, never parsed as truth.

Production requires an explicit persistent spool_dir (or FINEST_SPOOL_DIR). Without one, wrapping is pass-through and diagnostics reports production_requires_explicit_spool_dir; the SDK never substitutes a shared /tmp spool. Every simultaneously running process or replica needs a different persistent spool directory. A private owner record prevents live processes from sharing mutable journal and route state; dead same-host owners are reclaimed, while live, foreign-host, and malformed ownership fails closed.

Cross-language parity

finest.canonical implements RFC 8785 (JCS) canonical JSON and finest.ed25519 implements RFC 8032 verification, both in the standard library alone.

tests/test_golden_vectors.py loads packages/policy/golden/policy-vectors.json — generated by the TypeScript signer — and asserts that Python:

1. produces byte-identical canonical text for every vector;
2. reproduces every committed canonicalSha256;
3. verifies every valid and boundary vector;
4. rejects every tampered vector with the named rejection.

That is the proof two languages agree about what was signed, rather than merely looking similar. The vectors deliberately exercise UTF-16 key ordering (which is not code-point ordering once astral characters appear), minimal string escaping, the integer range where a JS double and a Python int stop agreeing, and the one key where null is legal.

Ed25519 is implemented here rather than taken as a dependency because finest's dependency set is a trust surface, and the alternative would pull a compiled crypto library in to verify a 32-byte key a few times a minute. It verifies only — there is no signing function and there must never be one, because a data-plane SDK that could mint a policy would defeat the signature entirely.

Diagnostics

finest.diagnostics()

Returns a support-safe snapshot: counts, reasons, route names, whether an API key is configured. It reports that a key exists, never its value. Nothing in this SDK prints, hashes or transmits key material.

Known limits of this version

• Streaming evidence is terminal. A create(..., stream=True) iterator is wrapped and journalled only when it finishes, fails, or is closed. Usage and served-model facts are accumulated when the provider emits them; omitted terminal usage remains an honest zero for reconciliation. A stream created but never consumed has no terminal attempt in this release.
• Observer tier only. Every attempt is arm: "passthrough" with policyId: null. This SDK cannot emit a challenger arm.
• Sync and async clients are supported; .stream() context managers are not intercepted — they pass through untouched.

Tests

python3 -m venv packages/sdk-python/.venv
packages/sdk-python/.venv/bin/pip install pytest httpx
packages/sdk-python/.venv/bin/python -m pytest packages/sdk-python -q

The suite imports no provider package. Client shapes are detected by their call paths, never by import openai, so the SDK works for a customer who has only one provider installed — or a vendored fork of either.

Licence

Apache-2.0. The data-plane SDK is open source on purpose: you should be able to read every line that touches your traffic.