TraceMotive

When an AI agent succeeds once and fails once, the difficult question is often not “did the runs differ?” but “where is the first difference that the evidence actually supports investigating?”

TraceMotive compares AI agent executions and shows where their observed behavior first diverges with enough evidence to support an investigation starting point. It runs locally, keeps the Collector and SQLite data on the machine, and presents uncertainty when the evidence cannot support a narrower claim.

TraceMotive does not claim that an observed divergence caused a later failure. It does not provide automatic RCA, replay, cloud observability, authentication, remote collectors, or broad framework support.

Try it locally in about three minutes

This is the normal installed-user path. It uses a deterministic local pair, so it needs no model provider, API key, Node.js, npm, cloud service, or external network request.

In the first terminal, install the server extra and start the loopback server:

python -m pip install "tracemotive[server]"
tracemotive serve

tracemotive serve binds to 127.0.0.1:8765 and serves the packaged UI and APIs. Keep this terminal running.

In a second terminal, seed the stable identified example:

tracemotive demo

Open the printed comparison URL. In the comparison, use the compact workflow:

• Look here — the first evidence-supported investigation starting point;
• What changed — the best supported behavioral description;
• Evidence — captured observations and their limitations;
• Next — open the left span, right span, or full comparison; and
• What TraceMotive does not know — the boundary between observation and explanation.

To see the uncertainty barrier for repeated members, run this in the second terminal instead:

tracemotive demo --scenario uncertain

Each invocation creates a fresh pair and leaves existing traces in place. The default demo is identified; the uncertain demo keeps repeated members unpaired when the evidence cannot establish their identity.

What the comparison means

TraceMotive reports observed evidence, not a diagnosis. A selected starting point means that the comparison found a supported place to begin investigating; it does not mean that TraceMotive knows the cause of a later failure.

The structured diff is deliberately conservative. It compares object keys in a deterministic order and emits bounded add, remove, and replace records at JSON Pointer paths. Arrays do not receive inferred identity or move semantics; complex arrays may fall back to a whole-array replacement. If capture is unavailable or redaction prevents a safe comparison, TraceMotive does not invent a detailed diff.

Evaluation scope

The public evaluation claim is limited to the current V03-10 adversarial corpus. In that 30-scenario corpus:

• 30 scenarios are mandatory;
• 15 have an expected confident meaningful-divergence answer;
• 14 have an expected supported investigation starting point;
• the false-confident meaningful-divergence target/result for a conforming outcome set is 0; and
• the false-confident investigation-starting-point target/result for a conforming outcome set is 0.

The corpus is intended to exercise ambiguity, incomplete traces, capture/redaction barriers, repeated tools, context-only changes, and structural divergence ordering. These are corpus-scoped oracle facts, not a universal accuracy guarantee, confidence percentage, causal claim, or independent benchmark.

The reproducible oracle checks are:

python -m unittest tests.test_divergence_evaluation -v
python -m tests.divergence_evaluation

The full report is the v0.3 divergence evaluation.

Supported integration

The validated framework integration is the public OpenAI Agents SDK adapter. The tested support range is openai-agents>=0.17,<0.18, with compatibility checks at versions 0.17.0, 0.17.4, and 0.17.8 for the adapter callbacks, span-data fields, processor registration, model settings, and example construction surface.

Install it separately when you want to instrument a real agent:

python -m pip install "tracemotive[openai-agents]"

The actual-agent example requires the model provider's credentials. It is not needed for the local deterministic demo. With the Collector already running:

python -m examples.openai_agents_example

local_only=True makes TraceMotive the only OpenAI Agents tracing processor in that process. It controls framework tracing processors; it does not make model traffic local. A provider request may still leave the machine.

LangGraph is not a validated integration in this checkout and is not part of the current support claim. It is conditional v0.4 design work only if its full public-runtime GO gate passes; otherwise it is deferred to v0.4.1. No other framework receives a support claim by analogy.

See the OpenAI Agents integration notes and the example README for the validated integration path.

Install a released package

The current release candidate distribution metadata is 0.4.0; this checkout is prepared for release but has not been published. For a normal installation:

python -m pip install tracemotive

For the local Collector and packaged UI:

python -m pip install "tracemotive[server]"

Normal installed users do not install Node.js, run npm, or run the repository bootstrap script.

Contributor setup from a fresh checkout

Contributor setup is separate from the installed-user path. Create and activate a virtual environment, install development dependencies, and then run the one canonical repository bootstrap command.

PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install "setuptools>=77,<84" wheel build
python -m pip install -e ".[server]"
python scripts/bootstrap.py

POSIX shells:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "setuptools>=77,<84" wheel build
python -m pip install -e ".[server]"
python scripts/bootstrap.py

python scripts/bootstrap.py runs the locked frontend install and package build from the repository root. It writes only disposable ignored generated UI assets. The development frontend is optional:

cd frontend
npm run dev

The production UI is embedded in the Python package and served by tracemotive serve; Node.js is not a runtime requirement for installed users.

For a maintainer-only direct Uvicorn factory check, the equivalent loopback command is:

python -m uvicorn tracemotive.collector:create_app --factory --host 127.0.0.1 --port 8765

Local SDK usage

TraceMotive and content capture are independently disabled by default. A minimal local trace is:

import tracemotive

tracemotive.configure(
    enabled=True,
    endpoint="http://127.0.0.1:8765",
    capture_content=False,
)

with tracemotive.trace("demo"):
    with tracemotive.span("work"):
        pass

tracemotive.flush()

The stable SDK surface remains configure, trace, span, and flush. Tracing failures, an unavailable Collector, and queue overflow do not fail the instrumented Agent execution.

Privacy and security boundary

• TraceMotive is disabled by default and includes no analytics or external TraceMotive telemetry.
• Content capture is independently disabled by default.
• The supported Collector transport is loopback-only.
• Framework adapters convert data into Canonical values before the shared privacy boundary redacts and sanitizes values before transport queue ownership.
• The Collector persists sanitized Canonical-derived data in local SQLite.
• Captured runtime content is untrusted data. The frontend renders it as data and does not execute embedded HTML, script, or arbitrary code.
• Tracing and sink failures remain isolated from the user's Agent execution.
• Provider traffic, such as an OpenAI model request, is separate from TraceMotive telemetry and may leave the machine.

Do not treat these statements as a formal security audit. For sensitive reports, read SECURITY.md and use GitHub Private Vulnerability Reporting rather than a public issue.

Compatibility and documentation layers

The repository intentionally keeps these layers separate:

The v0.4 API v3/v4 decision is contract-driven. /api/v3 remains unchanged. /api/v4 is needed because bounded operation diff records, explicit no-diff capture semantics, conservative array fallback, and cockpit action targets cannot be added to the existing v3 response without changing that contract. This is not API versioning caused by the package number.

Local validation

Python tests:

python -m unittest discover -s tests -v

Frontend tests and production build:

python scripts/bootstrap.py
cd frontend
npm test
npm run build

Packaging and installed-user checks are maintainer validation documented in the historical release-readiness checklist. They are separate from the normal installed-user quickstart.

Contributing

Read CONTRIBUTING.md before making changes. Keep changes within the current v0.4 Core scope while preserving the Frozen v0.1 contract. Do not infer support for deferred or conditional features from the long-term vision or from a passing unit test with a fake framework object.

Security

Do not report security vulnerabilities in public Issues or pull requests. Read SECURITY.md and use GitHub Private Vulnerability Reporting from the repository Security page.