llmkit-sdk 0.1.11

Local LLM cost estimates for existing Python SDK calls

llmkit-sdk wraps supported HTTP clients, reads token usage from provider responses, and estimates cost from a bundled pricing catalog. Local tracking does not require an LLMKit account or proxy.

pip install llmkit-sdk

Track an existing client

from llmkit import tracked
from openai import OpenAI

costs = []
client = OpenAI(http_client=tracked(on_cost=costs.append))

client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Explain CQRS."}],
)

print(f"${sum(item.total_cost or 0 for item in costs):.6f}")

The same transport can wrap an Anthropic client:

from anthropic import Anthropic
from llmkit import tracked

costs = []
client = Anthropic(http_client=tracked(on_cost=costs.append))

client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=512,
    messages=[{"role": "user", "content": "Explain event sourcing."}],
)

Estimate a completed response

from llmkit import estimate_cost

cost = estimate_cost(response)
print(f"~${cost.total_cost:.6f}")

LangChain callback

from llmkit.integrations.langchain import LLMKitCallbackHandler

handler = LLMKitCallbackHandler()
chain.invoke("Summarize this report", config={"callbacks": [handler]})
print(f"${handler.total_cost:.4f}")

Framework integrations are optional. Use a tested LLMKit extra where one is documented below; other integrations require their framework package separately.

PydanticAI gateway model

Use the native PydanticAI model interface to route requests through LLMKit for server-side budget admission, stable attribution, and gateway receipts:

pip install "llmkit-sdk[pydantic-ai]"

from pydantic_ai import Agent, ModelSettings, UsageLimits
from llmkit.integrations.pydantic_ai import gateway_model

model = gateway_model(
    "gpt-4.1-mini",
    session_id="release-review-42",
    workflow_id="release-review",
    agent_id="reviewer",
)
agent = Agent(model, model_settings=ModelSettings(max_tokens=512))

result = await agent.run(
    "Review this release candidate.",
    usage_limits=UsageLimits(request_limit=4, total_tokens_limit=8_000),
)

UsageLimits remains the in-run token and request guard. LLMKit gateway mode adds the shared, multi-run spend boundary and receipt. Hard budgets require an explicit positive output-token limit. Gateway-routed client-side function tools are supported but are not exact-effect-enforced; provider-managed tools, images, and file attachments fail closed when the gateway cannot prove a pre-dispatch cost ceiling. Transparent OpenAI SDK transport retries are disabled in gateway mode. Retry transient failures explicitly at the run boundary so every dispatch has a distinct budget reservation and receipt.

gateway_model() does not locally verify that a grant and terminal receipt match the exact request. Use the opt-in boundary model when the caller must withhold the model result until that proof is complete:

from pydantic_ai import Agent, ModelSettings
from llmkit.integrations.pydantic_ai import (
    PydanticAIBoundaryContext,
    gateway_boundary_model,
)

boundary_context = PydanticAIBoundaryContext(
    principal="reviewer@example.com",
    tenant="acme",
    workload="release-review",
    budget_scope="approved-budget-id",
    model_grant_resolver=resolve_model_grant,
    provenance="trusted",
)
model = gateway_boundary_model(
    "gpt-4.1-mini",
    context=boundary_context,
    runtime=boundary_runtime,
    provider="openai",
    settings=ModelSettings(max_tokens=512),
)

async with model:
    result = await Agent(model).run("Review this release candidate.")

Here, boundary_runtime and resolve_model_grant are application-owned. The resolver receives the exact serialized request action and must return its signed grant. The result is released only after the authenticated terminal receipt matches the request identity, budget, provider and model, response ID and body hash, and idempotency key. A denial stops before network dispatch; missing or mismatched evidence after dispatch produces uncertain. This boundary enforces non-streaming model calls only. Function tools need separate enrollment below. PydanticAI streaming and provider-managed tools remain explicitly uncovered, as do calls made directly through the wrapped model or OpenAI client.

PydanticAI function-tool boundary (experimental)

protect_function_tool() returns a native toolset for Agent(toolsets=[...]). It checks a signed grant for the exact tool name, version, call ID, validated JSON arguments, identity, policy, expiry, and budget scope before invoking the enrolled function. Arguments include native validated defaults and are copied before an asynchronous grant resolver runs.

from pydantic_ai import Agent, Tool
from llmkit.integrations.pydantic_ai import protect_function_tool

review_toolset = protect_function_tool(
    Tool(post_review_comment),
    context=boundary_context,
    runtime=boundary_runtime,
    grant_resolver=resolve_tool_grant,
    tool_version="1",
    effect_class="github.review_comment",
    acknowledgement=extract_review_acknowledgement,
)
agent = Agent(model, toolsets=[review_toolset])

The tool grant resolver receives the exact EffectAction and native RunContext. It returns a signed grant or None, synchronously or asynchronously. The acknowledgement callback must return an EffectAcknowledgement backed by the application's sink response. A successful function return alone does not settle the effect. Missing or invalid acknowledgement, cancellation, native timeout, and exceptions after invocation leave the receipt uncertain, never released.

Admission and dispatch have no intervening asynchronous step, so cancellation while resolving a grant leaves no reservation. Unlike the OpenAI Agents guardrail adapter, this toolset does not need a pending-admission finalizer. Use an application-owned context and runtime; the supplied HMAC and in-memory replay store prove an in-process lifecycle, not durable crash recovery or cross-process coordination.

Only explicitly wrapped native function tools with JSON-compatible validated arguments are covered. Approval-required or deferred tools, dynamic renames, custom toolsets, MCP tools, Python-object arguments, and direct calls to the original function are not covered. The coverage report declares enrollment, not an inventory of everything the Agent can execute.

The PydanticAI review example uses the same fake gateway and review sink as the OpenAI Agents example. From packages/python-sdk, with the pydantic-ai extra installed:

python ../../examples/pydantic_ai_boundary_review.py

Both native SDKs deny the poisoned review before the sink and join two approved model calls and one tool effect into nine receipt states. These local examples make no GitHub or hosted LLMKit request; they prove SDK wiring and the shared receipt contract, not deployment.

OpenAI Agents exact-effect boundary (experimental)

Use the opt-in boundary when an OpenAI Agents run must prove both model dispatch and function-tool effects. protect_function_tool() requires a signed grant for the exact tool, call ID, arguments, identity, policy, expiry, and budget scope before invoking the tool. GatewayBoundaryProvider uses the Agents ModelProvider seam to bind a separate grant to the exact serialized non-streaming model request before the HTTP transport sends it.

The model result remains withheld until an authenticated LLMKit receipt matches the request ID, identity, budget reservation, requested and last-dispatched provider and model, provider response ID, response-body hash, idempotency evidence, and terminal settled_actual state. Missing, expired, changed, or replayed grants stop before network dispatch. Cancellation, parsing failure, or missing terminal evidence after dispatch produces uncertain. The provider accepts one explicit provider and disables transparent OpenAI retries so one grant maps to one transport attempt.

pip install "llmkit-sdk[openai-agents]"

The local PR-review example runs the real Agents Runner against an in-process fake gateway. The poisoned review receives no tool grant and reaches the sink zero times. The approved review joins two model calls and one tool effect into three signed receipt chains. The fixture sends no GitHub or hosted LLMKit request, so it proves consumer wiring rather than hosted deployment.

From packages/python-sdk, the check takes a few minutes:

python -m venv .venv
.venv/bin/python -m pip install -e ".[openai-agents]"
.venv/bin/python ../../examples/openai_agents_boundary_review.py

On Windows, use .venv\Scripts\python.exe. A passing result reports zero poisoned sink calls, one approved sink call, two approved model requests, and nine approved receipt states in reserved, dispatched, settled order.

Wrap each Agents run in try / finally and call await release_pending_admissions(boundary_context) in the finalizer. This closes reservations when a run ends after the guardrail allows a tool but before the SDK invokes it. The context is single-run and rejects admissions after finalization.

Only function tools passed through protect_function_tool() and model calls routed through GatewayBoundaryProvider are enforced. Streaming model calls fail before dispatch because stream finality needs a separate evidence contract. The coverage report is declared scope, not runtime inventory. Approval-required function tools are rejected because OpenAI Agents 0.20 does not expose a rejection hook that can release a reserved grant. Unwrapped tools, hosted tools, hosted or local MCP, computer, shell, apply-patch, handoffs, agent-as-tool calls, realtime, direct clients, and background retries remain uncovered. The included HMAC authority and replay/lifecycle stores are local proof components, not a production key service or durable coordination layer.

Sessions and gateway mode

Use the hosted or self-hosted LLMKit gateway when you need shared budgets, request receipts, provider routing, or dashboard analytics. Hosted calls require an existing key. New hosted account creation and key management are temporarily unavailable.

from llmkit import LLMKit

client = LLMKit(api_key="llmk_your_key_here")
session = client.session()

completion, cost = session.chat(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Draft a release note."}],
)

print(f"${cost.total_cost:.4f} via {cost.provider}")

For an OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmkit.sh/v1",
    api_key="llmk_your_key_here",
)

Async client

from llmkit import AsyncLLMKit

client = AsyncLLMKit(api_key="llmk_your_key_here")
completion, cost = await client.chat(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Summarize this incident."}],
)

Accuracy boundary

• Local values are estimates derived from response usage metadata and the bundled pricing table.
• Provider invoice adjustments, account-specific discounts, and pricing changes may differ.
• Local tracking observes cost; budget rejection requires gateway mode.
• Streaming cost is final only after the stream completes and usage metadata is available.

LLMKit repository

The LLMKit monorepo also contains the Cloudflare Worker gateway, dashboard, TypeScript SDK, CLI, Vercel AI SDK provider, MCP server, database migrations, and deterministic budget-control fixtures.

License

MIT