Agent Trace Analysis Platform (atap)

The attribution & recovery layer on top of your existing agent observability stack (e.g. Langfuse) — find, explain, and fix LLM-agent failures, then write the answers back as scores

Python License: MIT

atap external evaluation on a live Langfuse instance

atap × Langfuse: pull live traces → analyze / classify / attribute → root-cause + confidence + blamed-step written back as Scores

atap pulls traces from Langfuse (or JSONL / OTel / Phoenix exports), locates the responsible agent and the decisive step of every failure, and writes the verdicts back as Langfuse scores.

  • 24 pluggable algorithms, 5 stages — one module each, YAML-composed, artifact-coupled
  • Deterministic offline mode — FakeLLM judge + fault-injected sandbox, zero network
  • Real LLMs — any OpenAI-compatible API, per-call audit log
  • Langfuse integration — atap langfuse-eval writes root-cause scores + blamed-step markers back onto your traces
  • Closed loop — one shared Hypothesis contract; recovered reruns re-enter analysis for verification

Results on real data

First external benchmark: Who&When (ICML 2025) — 184 real multi-agent failure trajectories, gold hidden from the judge, scored with the stock evaluator:

Stack (deepseek-v4-flash) step hit agent hit cost
all_at_once 33.2% 56.0%† $1.43 · 2.9h
all_at_once, thinking off 33.2% 54.3% $0.16 · 9min
binary_search 11.4% 34.2% $0.26 · 32min

vs the paper's GPT-4o baseline (Without-GT): step-level 2.9× on algorithm-generated (13.5% → 38.9%) and 5.9× on hand-crafted (3.5% → 20.7%) — on a cheaper model.

Quick start

pip install atap
atap demo    # offline end-to-end pipeline: FakeLLM judge, deterministic, zero network

Real-LLM runs, live-Langfuse round trips, the full head-to-head table, caveats and reproduction commands:

https://github.com/aaronlyt/agent-trace-analysis-platform

† hand-crafted agent figures corrected for a Magentic-One routing-label scoring artifact; the judge model differs from the paper's GPT-4o, so gains reflect prompt engineering + model — details and a controlled-ablation recipe in the repo's benchmark report.

Project links

Data verified by PyPI on Aug 31, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for aaronlyt from gravatar.com aaronlyt

Credits

Author: aaronlyt

GitHub Statistics

Data verified by PyPI on Aug 31, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.10

Provides Extra

llm langfuse dev

Tags

llm agents traces failure-attribution root-cause-analysis observability langfuse evaluation debugging

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atap-0.1.0.tar.gz (279.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

atap-0.1.0-py3-none-any.whl (225.9 kB view details)

Uploaded Python 3

File details

Details for the file atap-0.1.0.tar.gz.

File metadata

  • Download URL: atap-0.1.0.tar.gz
  • Upload date:
  • Size: 279.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for atap-0.1.0.tar.gz
Algorithm Hash digest
SHA256 dad716ed786cbcec211d121844c2b0ea714553bf67c16daf50d4a85e68e78e9f
MD5 eaf1880858d5b1a5f6a7ba641041e767
BLAKE2b-256 608505cc4266c0229372a639f4631b05dc718233e52f2e7cd1da568d0fba1c87

See more details on using hashes here.

Provenance

The following attestation bundles were made for atap-0.1.0.tar.gz:

Publisher: release.yml on aaronlyt/agent-trace-analysis-platform

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file atap-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: atap-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 225.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for atap-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b351b01161941f3f3e307c83725820a7e9b9cf89ada659ffb3eea4d1f572323f
MD5 98fa926e233eab8ecda75c98757f933e
BLAKE2b-256 2a271d7588940598467f620b147b44d6f6a0989d47b0b50d35660cfbb4ae521c

See more details on using hashes here.

Provenance

The following attestation bundles were made for atap-0.1.0-py3-none-any.whl:

Publisher: release.yml on aaronlyt/agent-trace-analysis-platform

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page