Skip to main content

CUTracer Python Module

Python tools for CUTracer trace validation, parsing, and analysis.

Overview

The cutracer Python package provides a comprehensive framework for working with CUTracer trace files. This module is designed to be:

  • Reusable: Import and use in your own Python scripts
  • Testable: Full unittest suite with real trace data
  • Type-safe: Type hints and mypy compatibility
  • Extensible: Plugin architecture for future enhancements

Installation

For Development

cd /path/to/CUTracer/python
pip install -e ".[dev]"

For Production Use

cd /path/to/CUTracer/python
pip install .

Features

Trace Validation (Current)

  • JSON Validation: Validate NDJSON trace files (mode 2) for syntax and schema compliance
  • Text Validation: Validate text-format trace files (mode 0) for format compliance
  • Cross-Format Consistency: Compare different trace formats for data consistency

Planned Features

  • Trace Parsing: Parse trace files into structured Python objects
  • Analysis Tools: Instruction histograms, performance metrics, trace comparison
  • Format Conversion: Convert between different trace formats
  • Compression Support: Handle zstd-compressed traces (mode 1)

Usage

CLI

After installation, the cutracer command provides a unified interface for tracing, querying, and analyzing CUDA traces:

# Trace a CUDA application (requires cutracer.so — see main README for build)
export CUTRACER_LIB_PATH=/path/to/CUTracer/lib
cutracer trace -i tma_trace -- python test.py

# Validate trace files
cutracer validate kernel_trace.ndjson

# Query and filter trace data
cutracer query trace.ndjson --filter "warp=24"

# Analyze traces
cutracer analyze warp-summary trace.ndjson

Python API

from cutracer.validation import (
    validate_json_trace,
    validate_text_trace,
    compare_trace_formats,
)

# Validate JSON trace
result = validate_json_trace("kernel_trace.ndjson")
if result["valid"]:
    print(f"✓ Valid JSON trace with {result['record_count']} records")
else:
    print(f"✗ Validation failed: {result['errors']}")

# Validate text trace
result = validate_text_trace("kernel_trace.log")
if result["valid"]:
    print(f"✓ Valid text trace")
else:
    print(f"✗ Validation failed: {result['errors']}")

# Compare two formats
result = compare_trace_formats("kernel_trace.log", "kernel_trace.ndjson")
if result["consistent"]:
    print("✓ Formats are consistent")
else:
    print(f"✗ Inconsistencies found: {result['differences']}")

Module Structure

python/
├── cutracer/                        # Main package
│   ├── __init__.py                  # Package entry point with version
│   └── validation/                  # Validation framework
│       ├── __init__.py              # Validation API exports
│       ├── schema_loader.py         # JSON Schema loader
│       ├── json_validator.py        # JSON syntax & schema validation
│       ├── text_validator.py        # Text format validation
│       ├── consistency.py           # Cross-format consistency checks
│       └── schemas/                 # JSON Schema definitions
│           ├── __init__.py
│           ├── reg_trace.schema.json
│           ├── mem_trace.schema.json
│           ├── opcode_only.schema.json
│           └── delay_config.schema.json
├── tests/                           # Unit tests
│   ├── __init__.py
│   ├── test_base.py                 # Base test class and utilities
│   ├── test_schemas.py              # Schema loading tests
│   ├── test_json_validator.py       # JSON validation tests
│   ├── test_text_validator.py       # Text validation tests
│   ├── test_consistency.py          # Consistency check tests
│   └── example_inputs/              # Real trace data for tests
│       ├── reg_trace_sample.ndjson
│       ├── reg_trace_sample.log
│       ├── invalid_syntax.ndjson
│       └── invalid_schema.ndjson
├── pyproject.toml                   # Modern Python project config
└── README.md                        # This file

Development

Running Tests

cd python/

# Run all tests
python -m unittest discover -s tests -v

# Run specific test file
python -m unittest tests.test_json_validator -v

Type Checking

cd python/
mypy cutracer/

Code Formatting

# From project root directory
./format.sh format

# Or manually with ufmt
ufmt format python/
usort format python/

Running All Checks

# Format code
./format.sh format

# Type check
mypy cutracer/

# Run tests
python -m unittest discover -s tests -v

Validation Details

JSON Trace Validation

The JSON validator checks:

  • Syntax: Valid JSON format on each line (NDJSON)
  • Schema: Correct field types and structure per JSON Schema
  • Required Fields: message_type, ctx, kernel_launch_id, trace_index, timestamp, sass, etc.
  • Register Values: Arrays of integers with proper format
  • CTA/Warp IDs: Valid integer ranges

Text Trace Validation

The text validator checks:

  • Format Patterns: Correct CTX/CTA/warp header patterns
  • Register Output: Proper hex format (e.g., Reg0_T00: 0x...)
  • Memory Access: Valid memory address patterns

Consistency Validation

The consistency validator compares:

  • Record Counts: Same number of records in both formats
  • Content Matching: Same kernel IDs, trace indices, SASS strings
  • Timestamp Order: Consistent ordering between formats

Trace Format Reference

JSON Format (NDJSON - Mode 2)

Each line is a JSON object with the following structure:

{
  "message_type": "reg_trace",
  "ctx": "0x58a0c0",
  "kernel_launch_id": 0,
  "trace_index": 0,
  "timestamp": 1762026820167834792,
  "sass": "LDC R1, c[0x0][0x28] ;",
  "pc": 0,
  "opcode_id": 0,
  "warp": 0,
  "cta": [0, 0, 0],
  "regs": [[0, 0, 0, ...]]
}

Text Format (Mode 0)

Human-readable format with CTX headers and register values:

CTX 0x58a0c0 - CTA 0,0,0 - warp 0 - LDC R1, c[0x0][0x28] ;:
    * Reg0_T00: 0x0000000000000000  Reg0_T01: 0x0000000000000000 ...

Contributing

  1. Install development dependencies: pip install -e ".[dev]"
  2. Make your changes
  3. Run tests: python -m unittest discover -s tests -v
  4. Run type checker: mypy cutracer/
  5. Format code: ./format.sh format
  6. Submit a pull request

License

MIT License - See LICENSE file for details.

Support

For issues and questions, please open an issue on the CUTracer GitHub repository.

Metadata

Release files for cutracer 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cutracer 0.4.0
File Size Uploaded
cutracer-0.4.0.tar.gz 160.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for cutracer 0.4.0
File Interpreter ABI Platform
cutracer-0.4.0-py3-none-any.whl Python 3 none any Details

Total release size: 308.9 kB

Release files / cutracer-0.4.0.tar.gz

Download URL cutracer-0.4.0.tar.gz
Size 160.2 kB
Tags Source
SHA-256 checksum
How to use checksums
021a7972bee9ac441430ae4c308f0ac4e0e36c891376de10a424769e06263e7b
BLAKE2b-256 checksum
How to use checksums
e5ada82ed85f15ab946e858cdd83ded8a38c67000d868948a7bfc85123cb78e1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.

Transparency log

Release files / cutracer-0.4.0-py3-none-any.whl

Download URL cutracer-0.4.0-py3-none-any.whl
Size 148.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2b1c7613c77e48ba9c4f60734b482222265aa2ab2296478c6b63b2baf24dd1d5
BLAKE2b-256 checksum
How to use checksums
a016beebba42cb0b831d6d80b8e5e60757e89afd1c89756db9d6556b15a964f0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

0.4.0 This release

2 release files

0.3.0

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page