DecimalAI Python SDK

Catch what your next agent change will break — a structural regression check against your recorded traces, built on manifest-aware versioning of your agent's tools, prompts, models, and skills. The open source SDK for DecimalAI.

Docs · Registry · Leaderboard · Changelog

Installation

pip install decimalai
# or
uv pip install decimalai

Requires Python 3.10+ (pip won't install current releases on older Pythons). The [adk], [pydantic-ai] and [all] extras need Python 3.11+: the google-adk and pydantic-ai-slim releases they install require it. On Python 3.10 pip does not refuse them; it installs an older decimalai instead (see below).

[langchain] installs on Python 3.10 with one limit there. asyncio on 3.10 cannot hand a run's config to an async LangGraph node, so a model that the node calls without passing config on, as create_agent's model node does, runs outside the run. Under ainvoke / astream those model calls route without the agent the trace is filed under, so that agent's skills and pins are not offered, and the trace records neither the calls nor their routing decisions. The SDK logs a warning the first time it happens. Sync .invoke() runs, nodes that pass config to the model, and every run on Python 3.11+ are unaffected; instrument(agent_name=...) at least routes the calls as the right agent.

Each framework extra supports the framework's latest release only, and floors it there.

The core install is deliberately thin (tracing, CLI, manifests, skills). Framework and provider adapters ship as extras — install the one matching your stack:

pip install "decimalai[langchain]"       # LangChain (add [langgraph] for LangGraph)
pip install "decimalai[openai-agents]"   # OpenAI Agents SDK
pip install "decimalai[all]"             # every framework adapter (not [evals]; see below)

Available extras: [langchain], [langgraph], [openai], [openai-agents], [llamaindex], [claude-agent-sdk], [pydantic-ai], [adk], [evals], [all].

[evals] installs litellm, which pins openai<3, while [openai], [openai-agents] and [pydantic-ai] need openai>=3. So [evals] is not part of [all], and it cannot be installed in the same environment as those three. CrewAI has no extra for the same reason: install crewai yourself, beside the adapters that run on openai<3.

When pip installs an older decimalai

Extras that cannot be installed together are not an error to pip. It goes back through older decimalai releases until one resolves, installs that, and prints nothing (uv pip install does the same). Two requests do this today, and both end on decimalai 0.13.9, the last release before the framework floors above, without this release's fixes:

• [evals] together with [openai], [openai-agents], [pydantic-ai] or [all], on any Python.
• [adk], [pydantic-ai] or [all] on Python 3.10.

Check the version you got, and put a floor on decimalai itself so pip names the conflict instead of falling back:

pip show decimalai                       # Version: 0.13.9 means pip fell back
pip install "decimalai[adk]>=0.14"       # fails with ResolutionImpossible rather than install 0.13.9

See it in 2 minutes

The regression demo seeds a reference agent (v1 → v2 + a trace corpus) into your workspace and runs the exact pipeline a real PR check runs. It needs a free API key — that's the only setup:

export DECIMAL_API_KEY="dai_sk_..."   # free key from app.decimal.ai/settings
decimalai demo regression             # → impact report: what your next change would break

The impact report it produces looks like this — captured with the SDK's decimalai regression-check against the seeded reference agent, trimmed:

  🔍 Agent Regression Check — [Demo] support-agent

  Manifest changes:
    🟡 tool_renamed — lookup_price
    🟢 tool_added — refund_order
    🟡 prompt_section_rewritten — system_prompt  [major, 88.6% changed]
    🔴 model_changed — gpt-4o-mini  (gpt-4o-mini-2024-07 → gpt-4o-mini-2024-09)
    🔴 tool_removed — compare_competitors

  Training-data policy (default):
    prompt_section_rewritten (major) → replay — need re-running first
    model_changed (major) → drop — excluded from training

Everything above comes from a seeded reference agent — illustrative, run it yourself. On your own instrumented agent, each change is additionally checked against your recorded traces — a high/medium/low blast radius per trace, not just the structural diff. The API key is needed because the demo seeds data into a workspace on the platform; decimalai demo reset removes it all.

These commands work without a DecimalAI key:

• decimalai skills pull <slug> — fetch any published SKILL.md to disk, no account.
• decimalai starter support --project ./support-agent — download a frozen, runnable Support project. Its Python standard-library runtime uses your OpenAI key locally and uploads no traces; python3 agent.py --check-setup verifies the files and --checks makes two task and two judge calls. The public builder downloads the same project without installing the SDK.
• The agentversion manifest flow (below) — diff and gate agent manifests fully locally.

There's a second demo for the registry side:

decimalai demo skills       # → tour the registry: security-scanned skills, benchmarkable with an open A/B spec

Browsing without an account? Explore the public skill registry — no signup required.

Quick Start

LangChain / LangGraph — Zero-Code Tracing and Skills

import decimalai

# That's it — chains, graphs and agents are traced, and the agent's skills are
# routed into every chat-model call. skills=False keeps it to tracing only.
decimalai.init(langchain=True, agent_name="support")

# Use LangChain as normal — nothing else changes
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

chain = ChatPromptTemplate.from_template("Tell me about {topic}") | ChatOpenAI(model="gpt-4o")
result = chain.invoke({"topic": "otters"})  # ← one trace, sent when the chain completes

A trace is sent when a chain, agent, or graph run completes. A bare llm.invoke("Hello!") with no surrounding chain never reaches that boundary, so it sends nothing — wrap the call, or see manual tracing for the per-call handler.

OpenAI Agents SDK

import decimalai

decimalai.init(openai_agents=True, agent_name="support")  # traces + skills menu + load_skill tool

init(langchain=True), init(openai_agents=True), init(adk=True) and init(crewai=True) install the same thing as that adapter's instrument(agent_name=..., enable_skill_loader=True). Use one or the other, not both.

Any Framework — Manual Tracing

import decimalai

decimalai.init()

@decimalai.trace(agent_name="my-agent")
def run_agent(query):
    msgs = [{"role": "user", "content": query}]
    resp = openai.chat.completions.create(model="gpt-4o", messages=msgs)
    decimalai.log_llm_call(
        model="gpt-4o",
        input=msgs,
        output={"content": resp.choices[0].message.content},
    )
    return resp.choices[0].message.content

Environment Variable Setup (No Code Changes)

export DECIMAL_API_KEY=dai_sk_...
export DECIMAL_AUTO_TRACE=langchain   # or "openai-agents", "adk", "crewai", "llamaindex", ...
# export DECIMALAI_SKILLS=0           # tracing only: no skill routing, nothing added to prompts
# Just run your app — tracing and skills activate on import
python app.py

DECIMAL_AUTO_TRACE=<framework> runs the matching decimalai.init() framework flag at import. On langchain, openai-agents, adk and crewai that includes the skill loader: each turn's message (on CrewAI, the task description) goes to the platform's router, and a skills menu (plus, on the injecting adapters, a skill body) is added to the prompt. Through 0.13.9 this path was tracing only. DECIMALAI_SKILLS=0 (or false/no/off) keeps it to tracing; it is the default of init(skills=...), so it applies to the flags in code too, and an explicit skills= argument wins over it.

What It Does

• Auto-tracing — Captures LLM calls, tool calls, and agent steps with zero code changes
• Agent versioning — Auto-detects tool schemas, prompts, models, and graph topology
• Change detection — Detects when your agent configuration drifts
• Inline evals — Run eval functions on every trace with @decimalai.evals.eval
• Built-in deterministic scores — completion, has_output, tool_compliance, latency, token_efficiency, attached by the LangChain adapter only: decimalai.langchain.instrument(...), disabled with instrument(..., builtin_evals=False). No other adapter attaches them today. These run in your process, not server-side — bare HTTP POST /traces does not auto-score.
• Batch evals — Run evals offline across historical traces
• Dataset pull — One-liner to download versioned training data: decimalai.pull_dataset("ds_abc", "./data.jsonl")
• HuggingFace Hub — Push datasets to HF Hub for instant Axolotl/Unsloth/TRL compatibility
• Fine-tuning — Launch fine-tuning jobs on OpenAI, Together.AI, or Gemini from the platform
• Skills management — Auto-discover SKILL.md files, sync to platform, install from registry
• OTel compatible — Export spans to any OpenTelemetry backend

How it fits together

The SDK is the client for the hosted platform and the reference implementation of two open specs — everything it captures is portable by design:

flowchart LR
  A[your agent code] --> SDK["decimalai SDK<br/>traces + manifests"]
  SDK --> API["DecimalAI platform<br/>registry + SkillScore"]
  SDK -.->|implements| AV[agentversion spec]
  SDK -.->|implements| SE[skillevaluation spec]
  RC["regression-check<br/>CI action"] --> API

• agentversion — the open manifest spec (versioning, diffing, compatibility)
• skillevaluation — the open A/B spec for measuring a skill's lift
• regression-check — the GitHub Action that runs the per-PR structural check

How does DecimalAI compare?

DecimalAI is not a tracing or eval-case platform — it's the structural layer that runs alongside one. LangSmith, Braintrust, and promptfoo watch your agent's outputs; DecimalAI versions its structure (tools + prompts + models + skills) and diffs every change against your recorded production traffic. Keep your eval tool; add the manifest layer under it.

The regression demo (decimalai demo regression) runs the real pipeline on a seeded reference agent — its numbers are illustrative; run it yourself, then instrument your own agent for real ones. Competitor capabilities checked against each tool's public docs, August 2026.

Skills

DecimalAI auto-discovers your existing SKILL.md files and provides observability — tracking which skills activate, how effective they are, and how they change over time.

Auto-Discovery (Bring Your Own Skills)

If you already have SKILL.md files (from npx skills add, your team's repo, or hand-written), the SDK discovers them automatically:

import decimalai
decimalai.init(api_key="dai_sk_...")

from decimalai.openai_agents import instrument
instrument()  # Scans .claude/skills/, .agents/skills/, etc. → syncs to dashboard

Supports 32 agent runtimes: Claude Code, Cursor, Copilot, Windsurf, Continue, and more.

Registry Search & Install

Find community skills and install them in one call:

from decimalai.skill_router import SkillRouter
router = SkillRouter(api_key="dai_sk_...")

# Search the public registry
results = router.search("code review security")

# Install a skill — a LINK into your workspace, tracking the author's updates
router.use("pdf")

# Write the files to disk, for runtimes that load SKILL.md themselves
router.export("pdf", agents=["claude-code", "cursor"])

# Take an editable copy, only if you intend to change it
router.fork("pdf")

Status & Update

# Check sync status between local files and platform
status = router.status()
# → {"synced": [...], "modified_locally": [...], "untracked": [...]}

# Pull upstream updates
router.update_skills()

Skill Delivery at Runtime (enable_skill_loader)

Discovery and sync (above) get skills into the platform. To get them into your agent's context at runtime, use the framework flag on init() (LangChain, OpenAI Agents, ADK and CrewAI turn the loader on with it), or enable the skill loader on your adapter's instrument():

import decimalai
decimalai.init()

from decimalai.openai_agents import instrument  # or .langchain / .adk / .crewai / .anthropic / .pydantic_ai
instrument(enable_skill_loader=True)

On CrewAI (crewai>=1.15.27) the loader rides CrewAI's own before_llm_call hook, and instrument() also turns on CrewAI tracing — the same exporter init(crewai=True) installs, once per process. Call it before the first kickoff(), because CrewAI copies its hook list into each agent executor when it builds one:

import decimalai
decimalai.init()

from decimalai.crewai import instrument
instrument(agent_name="support", enable_skill_loader=True)

config = decimalai.load_agent("support")
agent = Agent(role="Support", goal="Resolve the ticket", backstory=config.system_prompt or "", llm=llm)
Crew(agents=[agent], tasks=[Task(description=question, expected_output="the answer", agent=agent)]).kickoff()

Tracing needs the OpenInference CrewAI instrumentor (pip install openinference-instrumentation-crewai) plus the one for your model's provider SDK (openinference-instrumentation-openai, …), exactly as for init(crewai=True).

You can read each run's trace as the LangChain and ADK adapters let you: instrument(..., on_trace=fn) calls fn once per CrewAI run, failed runs included, with a copy of the finished trace (a RunTrace with its id, parent_trace_id, routing id and delivered skills) before it is exported. An fn that raises is logged and changes nothing else. A run made inside decimalai.start_trace() is filed as that trace's child (parent_trace_id), with or without an observer. CrewAI's trace is built from OpenTelemetry spans that wait in a batch queue, so call decimalai.flush() before a short-lived process reports or exits: it exports those spans first, waiting at most five seconds, and then sends.

With the loader on, the router adds a ranked menu of relevant skills to the prompt (one short row per skill: name + when to use it). A menu row alone is only an offer — the skill's actual content (its body) reaches the model through one of three delivery mechanisms:

• Body injection — decimalai.init(inject_skill_body=...) (or DECIMALAI_INJECT_SKILL_BODY=0/1) injects the top-routed skill's full body into the prompt. Bodies go in complete or not at all: one that does not fit the per-turn token budget is left out, never trimmed. A body is injected only when the platform says it ranked the result by this turn's message (query_ranked), so a "thanks!" gets the menu alone. A degraded result still counts when it is ranked, such as lexical-only retrieval while embeddings are down. Two bodies go in ranked or not: the priority_skills the platform offered, and the agent's only skill when its whole menu is one skill (which therefore reaches small talk too). The flag is tri-state, not opt-in: left unset it defaults on for the adapters where injection is the only body channel (langchain, anthropic, adk, crewai — none of them register a load_skill tool) and off for openai_agents and pydantic_ai, which fetch on demand rather than double-deliver (turn the load_skill tool off and injection takes over there too). An explicit True/False always wins.
• load_skill tool — on adapters that own their tool loop, a load_skill tool registers automatically whenever the loader is enabled, so the model can fetch an offered skill's body mid-turn. Each load names the run's routing decision (routing_id), and the platform decides what to serve: any skill your org owns, has installed or has assigned to the agent (including one the menu cap left off this turn's menu), and a skill reachable only through the public registry just when the run was offered it, so a prompt injection can't load an arbitrary registry skill by name. The SDK refuses no name itself; a load from a router that has never routed (a hand-built menu, a direct SkillRouter.load_skill() call) names no decision and is not confined. If another tool in the run is already called load_skill (an OpenAI SandboxAgent's native skills, or your own tool on Pydantic AI), DecimalAI's is exposed as decimalai_load_skill instead and the menu names it. On by default; kill switch: decimalai.init(load_skill_tool=False) or DECIMALAI_LOAD_SKILL_TOOL=0. It is not an instrument() parameter on these adapters.
• Export to disk — for runtimes that natively load skills from files (Claude Code, Cursor, ...), write them out with router.export(...) or decimalai skills export and let the runtime deliver them. Export takes no copy and needs no fork; you can export a skill you only linked.

Delivery support differs per adapter — the asymmetry is structural (load_skill needs a tool loop the adapter controls):

Honest-measurement note: menu-only (loader on, no body injection, no load_skill tool) means the model sees that a skill exists but never its content. Usage from that channel counts as offered, not activated — don't expect activation stats from prompt-injection-only setups.

Other Ways to Use Skills (No SDK)

Every published skill is reachable without installing anything:

• Web copy-paste — open any skill's scorecard page on app.decimal.ai/skills, hit Copy SKILL.md, and paste it into your repo.
• Raw URLs — https://app.decimal.ai/s/<slug>/SKILL.md serves the raw markdown (version-pinned: /s/<slug>@<version>/SKILL.md); https://app.decimal.ai/s/<slug>.json serves machine-readable metadata (lift summary, benchmark models, trust/safety bands); https://app.decimal.ai/llms.txt indexes the registry for agents.
• CLI pull (no account) — decimalai skills pull <slug> writes just the file to disk; no fork, no signup. (decimalai skills install <slug> forks + syncs if you do have a key.)
• MCP server — search and read skills from any MCP client (pip install decimalai-mcp).

See the full SDK Skills Reference for all methods.

Datasets & Training

Pull Training Data

import decimalai
decimalai.init()

# Pull the latest version to a local file
result = decimalai.pull_dataset("ds_abc123", "./training_data.jsonl")
print(f"Wrote {result['row_count']} rows")

# Pull a specific version
result = decimalai.pull_dataset("ds_abc123", "./data.jsonl", version="v2")

Push to HuggingFace Hub

# Push to HF Hub — instantly loadable by Axolotl, Unsloth, TRL
result = decimalai.push_to_hub("ds_abc123", "my-org/support-agent-sft")

# Now usable everywhere:
# from datasets import load_dataset
# ds = load_dataset("my-org/support-agent-sft")

Load as HuggingFace Dataset (In-Memory)

# Skip files — load directly into your training script
ds = decimalai.load_hf_dataset("ds_abc123")
# → Dataset({features: ['messages'], num_rows: 500})

CLI

# Pull latest version
decimalai datasets pull ds_abc123 -o ./training_data.jsonl

# Pull specific version as Parquet
decimalai datasets pull ds_abc123 -o ./data.parquet --version v2

# Push to HuggingFace Hub
decimalai datasets push-to-hub ds_abc123 my-org/support-agent-sft

Fine-Tuning Providers

Supported Frameworks

Tracing a direct LLM SDK with no agent framework? Use the provider flags: init(openai=True), init(anthropic=True), or init(google=True).

Examples

See the examples/ directory for runnable notebooks with Open in Colab badges:

Notebook	Description	Colab
Quickstart	Full version-aware loop — no LLM key needed
LangChain	Instrument a LangChain agent
OpenAI Agents	Instrument an OpenAI Agents app
Evaluations	Run built-in evaluators on traces
Datasets	Build SFT training datasets
Pull & Push	Pull datasets locally, push to HuggingFace Hub	—
Version-Aware Loop	Detect manifest changes and impact

Open standard: agentversion

The manifests this SDK captures are agentversion manifests — the open spec for agent versioning, diffing, and compatibility decisions that DecimalAI is built on. export_manifest hands a captured manifest to the OSS tooling, so you can diff and gate it in CI with no platform account:

import decimalai
from decimalai.schema.manifest import extract_from_config
from agentversion.diff import diff_manifests              # pip install agentversion
from agentversion.compatibility import classify_compatibility

snap = extract_from_config(agent_name="support-agent", prompts={...}, models={...})
manifest = decimalai.export_manifest(snap)                # → an agentversion manifest dict
print(classify_compatibility(diff_manifests(last_prod, manifest)).recommended_decision)

You can reproduce the platform's diffs and verdicts entirely outside DecimalAI — the SDK is the convenience layer over the open standard.

FAQ

Development

pip install -e ".[dev]"
python -m pytest tests/ -q --ignore=tests/test_langchain_compat.py
ruff check decimalai/ --select I,E,W,F --ignore E501,E402,F821,F841
git grep -nE 'decimal[-_]ai'   # must find nothing — the package is 'decimalai', no separator

Run these before opening a PR. CI runs the same commands, plus the LangChain compatibility matrix in tests/test_langchain_compat.py. See AGENTS.md if you're pointing an AI coding agent at this repo.

Documentation

Full docs at docs.decimal.ai

License

MIT — see LICENSE for details.

Docs · Registry · SDK · Specs: agentversion · skillevaluation · regression-check

Optional runtime evidence

Authenticated applications can enable bounded metadata evidence with decimalai.init(api_key=..., evidence=True, evidence_directory="/private/evidence"). It keeps a separate durable local spool and uploads only when the server advertises /api/v1/evidence/capabilities with the supported after-commit receipt contract. Collection is off by default; offline, manifest-only and keyless starter work gains no uploads. Credentials, prompts, answers and skill bodies are never evidence payloads. Missing capability or a transport failure retains pending events without retrying your model work. Uploaded client reports do not themselves verify source activation, business publication, retirement or account deletion.