tensor-spec 0.1.2

Framework-agnostic deep learning API quality verification tool

tensor-spec

Overview

tensor-spec is a framework-agnostic deep learning API quality verification tool. It compares tensor operation precision across deep learning frameworks (currently PaddlePaddle vs PyTorch) using process isolation and CUDA IPC zero-copy GPU data sharing.

Features

• Framework-agnostic -- Pluggable backend architecture supporting PaddlePaddle and PyTorch
• Process isolation -- Each framework runs in its own subprocess to avoid CUDA/import conflicts
• CUDA IPC zero-copy -- GPU tensor sharing via CUDA IPC handles (1000x faster than CPU roundtrip)
• Secure config parsing -- Whitelist recursive descent parser (no eval())
• Constraint-driven data generation -- Generate test tensors from specs with composable constraints
• Per-API tolerance -- Configurable atol/rtol with TOML-based overrides per operator
• Structured logging -- JSON Lines output for machine-parseable test results
• CLI interface -- check, bench, doctor, setup, migrate, pack/unpack commands via Typer

Installation

pip install tensor-spec

Or with uv:

uv add tensor-spec

Quick Start

# Check a single case on one backend
tensor-spec check 'abs(x=Tensor.float32((2, 3)))' --target backend=torch,python=/path/to/python

# Cross-framework comparison (auto-selects reference)
tensor-spec check cases/ --target torch-cu124 --target paddle-cu124

# e2e benchmark
tensor-spec bench --mode e2e cases/ --target torch-cu124 --target paddle-cu124 --baseline torch-cu124

# NCU benchmark (artifacts default to ./.tensor-spec/bench-artifacts)
tensor-spec bench --mode ncu cases/ --target torch-cu124 --target paddle-cu124

# Dry-run validation (no execution)
tensor-spec check cases/ --target torch-cu124 --dry-run

# Environment diagnosis
tensor-spec doctor

Development

Prerequisites

• Python >= 3.12
• uv for dependency management
• just for task running

Setup

just install

Common Commands

just            # List all available commands
just format     # Format code (Python + justfile)
just lint       # Run linter
just check      # Type checking (pyright)
just test       # Run all tests
just cov        # Run tests with coverage
just docs       # Serve docs locally
just ci         # Full CI check (format + check + lint + cov)

Architecture

The project follows a 10-layer modular architecture:

Layer	Module	Responsibility
1	spec	Pure data structures for tensor specifications
2	config	Secure whitelist recursive descent parser
3	datagen	Constraint-driven numpy array generation
4	backend	Pluggable framework backends (Paddle, PyTorch)
5	mapping	API name/argument mapping between frameworks
6	compare	Tensor comparison with configurable tolerance
7	tester	Test execution modes (in-process, isolated)
8	worker	Subprocess management and IPC protocol
9	ipc	SharedMemory (CPU) and CUDA IPC (GPU) transport
10	cli	Typer-based CLI entry point

Documentation

For detailed usage and API documentation, please visit our documentation site. For CLI design details, see CLI Specification.

Contributing

We welcome contributions! Please see CONTRIBUTING.md for development setup and contribution guidelines.