xprobe
Find out why your CPU, Python, or CUDA process is slow — in two commands.
A bounded Linux profiler for AI coding agents and the humans working with them:
eBPF + perf + NVIDIA CUPTI, one CLI, strict JSON, no daemon.
Real numbers from the v0.6.0 release gate on an RTX 3060: a PyTorch CUDA workload and a request → launch → kernel path.
What it does
Point xprobe at a running process. It watches one bounded window and tells you where the time went — on-CPU hotspots, threads waiting for a CPU or blocked, container CPU throttling, hot syscalls, GPU kernels, and GPU clock limits — ranked by impact, each with a ready next measurement. The second command then measures that exact latency: a function, a syscall, a CUDA API call, a kernel, a copy, an NVTX range, or a whole request → launch → kernel path from a single capture.
- Built for AI agents. Ships as an Agent Skill
for Claude Code, Codex, and Cursor. Compact JSON with a
good/cautionquality verdict, so an agent can act on results without reading 100 KB of events. - CPU, Python, and GPU in one tool. Sampled native and CPython stacks, eBPF uprobes, syscalls, tracepoints, CPython GC, and CUDA 12/13 activity via CUPTI, correlated with explicit confidence.
- Bounded by design. Every capture is limited by samples or duration, timeout, and capacity; there is no daemon; CUDA injection is disclosed before it happens; the raw events are always kept for offline re-analysis.
- Honest about what it knows. Sampled shares are estimates, temporal matches are labeled heuristic, and anything a capture cannot answer is listed as not covered.
Install
Install the version-matched Skill with the open Agent Skills CLI. This is the only installation action required from the user:
npx skills@1 add \
https://github.com/itdevwu/xprobe/tree/v0.6.2/skills/xprobe-measure-latency \
--global
The installer detects Codex, Claude Code, Cursor, and other compatible agents.
When invoked, the Skill checks for the matching xprobe CLI and installs or
repairs it under a writable prefix before profiling. It can then diagnose and
adjust path, permission, NVIDIA, CUDA, or CUPTI problems from live evidence.
Node.js is only needed for Skill installation, not for xprobe itself.
To install only the CLI, use pip or the release archive:
pip install xprobe
See Installation for the release archive, SPDX SBOMs, and GitHub-hosted build attestation verification.
Measure
Triage an unknown process over one representative window. Findings are ranked by their estimated share of the window, and each carries a ready next step:
xprobe measure --pid 4242 --triage --duration-ms 2000 > triage.json
jq '.findings[0].next.spec' triage.json > next.json
xprobe measure --spec next.json
Measure a known boundary directly, or several related boundaries from one capture with a measurement graph:
xprobe measure --pid 4242 \
--from 'cuda:runtime_api:cudaLaunchKernel:exit' \
--to 'cuda:kernel_start:name~flash.*' \
--match exact --samples 100
Output is JSON whenever stdout is not a terminal. Results carry latency
statistics, a good/caution quality verdict, rank exemplar pairs, and the
path of the bounded event capture, which measure --input re-correlates
offline. See the CLI contract for every mode.
Public CLI
| Command | Purpose |
|---|---|
doctor |
Report local eBPF, ptrace, NVIDIA, CUDA, and CUPTI capabilities |
discover |
List NVML-confirmed CUDA context holders under a process-tree root |
validate |
Check an event pair, graph, or inventory mode without attaching |
measure |
Triage, inventory, or correlate bounded live or recorded evidence |
measure --pid automatically loads the matching CUDA 12 or CUDA 13 CUPTI Agent
when a selected endpoint requires it. It reports the target mutation on stderr
and in JSON, disables collection afterward, and leaves the shared object mapped
for safe reactivation. NVTX ranges are the exception: set
NVTX_INJECTION64_PATH before the target's first NVTX call because online
attach cannot retrofit an initialized NVTX dispatch.
Support
| Surface | Current support |
|---|---|
| OS/architecture | Linux x86_64, glibc 2.34 or newer |
| Triage | One window of thread time, cgroup throttling, CPU samples, syscalls, NVML health, and CUDA kernels |
| Measurement graphs | Up to eight edges from one capture |
| CPU inventory | PID-scoped sampled user stacks with native ELF and CPython perf-map symbols |
| Host events | ELF functions, named syscall/tracepoint boundaries, and CPython GC USDT |
| Coarse aggregation | Bounded syscall lifecycle and CUDA kernel/memcpy/memset groups, with pageable/pinned copies |
| CUDA callbacks | Runtime and Driver API entry/exit |
| GPU activity | Kernel, memcpy, and memset start/end |
| Application ranges | Bounded ASCII NVTX thread and process ranges |
| CUDA/CUPTI | 12.x and 13.x with automatic major selection |
| Correlation | exact, first-after, nearest, stack-nested, stream-order |
| Online injection | same mount namespace; ptrace permission required |
GPU-to-GPU durations remain available across the supported majors. Cross
CPU/GPU subtraction requires the Agent's runtime alignment check to pass;
otherwise measure returns CLOCK_ALIGNMENT_FAILED rather than treating an
offset CUPTI clock as host monotonic.
Documentation
- Installation
- Architecture
- CLI contract
- CUPTI agent and injection
- Development and hardware tests
- Releasing
- Agent integration
- Public JSON schemas
Source builds and hardware tests are documented under Development. Implemented behavior is defined by code, tests, and schemas. Licensed under Apache-2.0.
Metadata
Release files for xprobe 0.6.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| xprobe-0.6.2-py3-none-manylinux_2_34_x86_64.whl | Python 3 | none | Linux glibc 2.34+ x86-64 | Details for xprobe-0.6.2-py3-none-manylinux_2_34_x86_64.whl |
Release files / xprobe-0.6.2-py3-none-manylinux_2_34_x86_64.whl
| Download URL | xprobe-0.6.2-py3-none-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | Linux glibc 2.34+ x86-64 Python 3 |
|
SHA-256 checksum How to use checksums |
523edad2b6227462a39983410547704dc25389be206d53b220f51686a3444a8f
|
|
BLAKE2b-256 checksum How to use checksums |
ea1bff971c534711265f301c7eb0e9cb323d2ce5d99c7949cc0e1eb1eca7efac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.
Transparency logSecurity for xprobe 0.6.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
1 file in this release has been uploaded with provenance metadata.
Attestation for xprobe-0.6.2-py3-none-manylinux_2_34_x86_64.whl
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 21, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.