slowai 0.5.0

One command to find why your PyTorch model is slow — and fix it.

slowai

One command to find why your PyTorch model is slow — and fix it.

slowai diagnoses which performance regime your workload is stuck in (compute-bound, memory-bound, or overhead-bound), prescribes the right fix, auto-applies it, and proves the speedup with before/after measurements. No guesswork. No manual profiler interpretation.

$ slowai fix model.py

==============================================================
  BASELINE: model.py: COMPUTE_BOUND (confidence: 0.85)
  wall time: 7.523s
==============================================================

  Tried 4 remedies:

  1. [10.00x] bf16_autocast  ** BEST **
     Run under bfloat16 automatic mixed precision
     7.523s >>> 0.752s
     regime: compute (confidence: 0.85)

  2. [6.32x] tf32_tensor_cores
     Enable TF32 tensor cores (~2x matmul throughput on Ampere+)
     7.523s >>> 1.191s

  3. [6.22x] high_matmul_precision
     Set float32 matmul precision to 'high'
     7.523s >>> 1.210s

  4. [1.31x] cudnn_benchmark
     Enable cuDNN auto-tuner for conv kernels
     7.523s >>> 5.719s

--------------------------------------------------------------
  WINNER: bf16_autocast
  7.523s >>> 0.752s  (10.00x, +900% faster)
  How: Run under bfloat16 automatic mixed precision
--------------------------------------------------------------

Why this exists

Every deep learning workload is stuck in one of three performance regimes (Horace He, 2022):

Regime	What's happening	Wrong fix = no speedup
Compute-bound	GPU is saturated doing math (matmuls, convolutions)	Fusing ops won't help — the math itself is the bottleneck
Memory-bound	GPU is waiting for data (pointwise ops, activations)	Smaller model won't help — you need less data movement
Overhead-bound	GPU is idle waiting for Python/dispatcher (tiny ops)	Lower precision won't help — you need fewer, bigger ops

The fix for each regime is different. Applying a compute-bound fix to a memory-bound workload does nothing. Engineers waste hours in profiler UIs figuring this out manually.

slowai does it in one command.

How it works

slowai diagnose model.py           # Classify the regime + prescribe fixes
slowai fix model.py                # ^ plus auto-apply fixes and measure speedup
slowai fix model.py --export       # ^ plus save winning config as slowai_config.py

Under the hood:

1. Profile — Runs your workload under torch.profiler with CUDA timing, warmup pass, and op-level statistics
2. Classify — A heuristic classifier analyzes op shares (matmul, normalization, pointwise, tiny-op fraction) to determine the dominant regime
3. Prescribe — Returns a ranked list of fixes for that regime, cheapest first
4. Remediate — Auto-applies each applicable fix (TF32 tensor cores, bf16/fp16 autocast, cuDNN benchmark, matmul precision), re-profiles, and ranks by measured speedup

No code changes required. Remedies are environment-level transforms — they modify PyTorch's runtime settings, not your model code.

Export to production

The --export flag saves the winning remedy as a drop-in Python module:

slowai fix model.py --export
# Creates slowai_config.py in the current directory

Then in your production code:

import slowai_config
slowai_config.apply()  # Set globally before your model runs

# Or as a context manager:
with slowai_config.optimized():
    model(data)

The exported config includes the exact PyTorch settings, speedup metadata, and both a global apply() function and an optimized() context manager. Zero dependencies beyond PyTorch.

CI/CD mode

Catch performance regressions on every commit:

# Fail if no remedy achieves at least 1.5x speedup
slowai fix model.py --ci --threshold 1.5

# Returns exit code 0 (pass) or 1 (fail)
# Outputs JSON for pipeline consumption
echo $?

Combine with --export to auto-generate optimized configs in your pipeline:

slowai fix model.py --ci --threshold 2.0 --export slowai_config.py

Drop the included examples/github-actions-ci.yml into your repo at .github/workflows/perf-check.yml for a ready-made GitHub Actions workflow that checks performance, uploads reports, and comments on PRs.

Benchmarks

Tested on NVIDIA Jetson Orin Nano Super (Ampere GPU, 1024 CUDA cores, 8GB unified RAM, JetPack 6.2, CUDA 12.6, PyTorch 2.8.0). 27 workloads across 18 industry verticals — the most comprehensive edge AI performance benchmark suite available.

Synthetic workloads (regime validation)

Workload	Regime	Baseline	Best remedy	After	Speedup
Dense GEMM (4096x4096)	Compute	7.523s	bf16_autocast	0.752s	10.00x
Pointwise chain (8192x8192)	Memory	2.400s	tf32_tensor_cores	0.575s	4.17x
Tiny ops (5000 micro-ops)	Overhead	3.281s	tf32_tensor_cores	1.141s	2.88x

Production models — standard architectures

Workload	Industry	Baseline	Best remedy	After	Speedup
MobileNetV2	Mobile / Edge	1.778s	cudnn_benchmark	1.681s	1.06x
ResNet-50	Classification	2.105s	bf16_autocast	1.969s	1.07x
EfficientNet-B0	Drones / Aerospace	1.445s	cudnn_benchmark	1.404s	1.03x
R3D-18 (video)	Surveillance / Defense	6.032s	bf16_autocast	3.443s	1.75x
Transformer (12L/768d/12H)	NLP / LLMs	6.207s	bf16_autocast	2.060s	3.01x

Production models — industry-specific pipelines

Workload	Industry	Baseline	Best remedy	After	Speedup
Underwater AUV (sonar + camera + nav)	Oil & Gas / Navy	2.232s	cudnn_benchmark	0.122s	18.34x
LiDAR 3D point cloud (PointNet-style)	Autonomous vehicles	2.288s	bf16_autocast	0.138s	16.57x
Agriculture drone (multispectral + NDVI)	Precision agriculture	—	tf32_tensor_cores	—	13.78x
Pose estimation (FPN + PAF, multi-person)	Retail / AR-VR / Sports	2.394s	bf16_autocast	0.287s	8.34x
Satellite imaging (change detection + priority)	Space / Defense	—	bf16_autocast	—	7.72x
Robotics pick-and-place (RGB-D + 7-DOF)	Industrial robotics	—	cudnn_benchmark	—	7.39x
GNN smart grid (message-passing + pooling)	Energy / Telecom	2.569s	tf32_tensor_cores	0.453s	5.67x
Medical imaging (DenseNet + multi-task)	Healthcare	—	bf16_autocast	—	5.54x
1D ConvNet (signal processing)	Navy radar / sonar	2.990s	bf16_autocast	0.757s	3.95x
Time Series Transformer	Predictive maintenance	3.814s	bf16_autocast	1.072s	3.56x
Edge diffusion (UNet denoiser, 128x128)	Generative AI on device	2.904s	bf16_autocast	0.918s	3.17x
Fly-by-wire control (sensor + transformer)	Aviation / eVTOL	—	tf32_tensor_cores	—	3.08x
Cybersecurity anomaly (flow transformer)	Network defense / SOC	3.749s	tf32_tensor_cores	1.308s	2.87x
Speech-to-text (Whisper-style encoder-decoder)	Consumer / Accessibility	—	tf32_tensor_cores	—	2.50x
RL policy network (LSTM + multi-modal, 200Hz)	Industrial robotics / Logistics	4.370s	tf32_tensor_cores	2.071s	2.11x
Mamba SSM (selective state-space, 4-layer)	Telecom / IoT	30.582s	tf32_tensor_cores	27.448s	1.11x
DeepLabV3 (MobileNetV3)	Autonomous driving	4.019s	bf16_autocast	3.623s	1.11x
Detection + Segmentation pipeline	Autonomous driving	5.744s	bf16_autocast	5.289s	1.09x
SSD-Lite (MobileNetV3)	Autonomous driving	1.918s	cudnn_benchmark	1.816s	1.06x

What the results tell you

Massive gains (5-18x) on custom multi-stream pipelines — AUV sensor fusion, LiDAR 3D processing, agriculture multispectral, pose estimation, GNN smart grid. These architectures use unique compute patterns (point clouds, multi-modal fusion, feature pyramids, scatter/gather ops) that PyTorch doesn't optimize by default.

Strong gains (2-5x) on transformer-based models and recurrent policies — cybersecurity flow analysis, speech-to-text, time series, BERT, RL policy networks, edge diffusion. Mixed precision and TF32 dramatically reduce matmul cost.

Modest gains (1-1.1x) on already-optimized architectures and sequential workloads — MobileNet, EfficientNet, SSD-Lite, Mamba SSM. Mobile architectures use depthwise separable convolutions that are already fast; sequential scan models are overhead-bound and need torch.compile (V5).

The real value is that slowai finds the right fix automatically — cuDNN benchmark wins for convolution-heavy models, bf16 autocast wins for matmul-heavy architectures, TF32 wins for transformer workloads. Different models, different winners, zero guesswork.

Industries covered

Autonomous vehicles, aviation/eVTOL, oil & gas, Navy/defense, marine science, space, healthcare, industrial robotics, precision agriculture, cybersecurity, consumer/AR-VR, sports analytics, retail, generative AI, predictive maintenance, energy/smart grid, telecom/IoT, warehouse logistics.

Installation

git clone https://github.com/ricojallen37-sketch/slowai.git
cd slowai
pip install -e .

Requires Python 3.10+ and PyTorch >= 2.1 with CUDA support.

Writing a workload

slowai profiles any Python script that exposes a main() function:

# my_model.py
import torch
from torchvision.models import resnet50

model = resnet50().cuda().eval()
data = torch.randn(8, 3, 224, 224, device="cuda")

def main():
    with torch.no_grad():
        for _ in range(30):
            model(data)

slowai fix my_model.py

Architecture

slowai/
  schema.py      # Regime enum, Diagnosis dataclass — the product thesis in types
  profiler.py    # torch.profiler wrapper → ProfileResult (op stats + wall time)
  diagnose.py    # Heuristic classifier → Diagnosis (regime + confidence + prescriptions)
  remediate.py   # Auto-fix engine → FixReport (before/after speedup per remedy)
  cli.py         # CLI entry points: diagnose, fix

The classifier is a pure function of profiler output — no torch dependency, fully unit-testable. The remediate engine applies fixes as environment transforms (global flags, autocast context managers) so it never modifies user code.

What's different

Other tools in this space are profiler UIs that show you data and leave interpretation to you. slowai is the only tool that goes from raw workload to regime classification to ranked prescriptions to auto-applied fixes to measured speedup in a single CLI command.

Tool	Profiles	Classifies regime	Prescribes fixes	Auto-applies	Measures speedup
PyTorch Profiler	Yes	No	No	No	No
NVIDIA Nsight	Yes	No	No	No	No
torch.utils.bottleneck	Yes	No	No	No	No
DeepSpeed Flops Profiler	Yes	No	No	No	No
slowai	Yes	Yes	Yes	Yes	Yes

Roadmap

• V1 (shipped) — Profile + classify regime for synthetic workloads
• V2 (shipped) — Noise filtering (sync ops, init ops), normalization-aware classification, real model support
• V3 (shipped) — Auto-remediate: apply fixes and measure before/after speedup
• V3.1 (shipped) — --export flag: save winning config as production-ready Python module
• V3.2 (shipped) — --ci mode: CI/CD integration with threshold-based pass/fail, GitHub Actions workflow
• V4 (shipped) — channels_last memory format remedy, Jetson power mode detection, numerical accuracy validation, 27 workloads across 18 industries (added GNN/smart-grid, Mamba/SSM, RL policy net)
• V5 (shipped) — TensorRT backend via torch.compile, torch.compile inductor for overhead-bound workloads, INT8 dynamic quantization, DLA detection, transform_main remedy architecture for JIT compilation
• V6 (next) — TensorRT static engine export, INT8 calibration-based quantization, DLA offloading, streaming inference mode, multi-GPU support

Built by

Rico Allen — @ricojallen37-sketch

Built and tested on NVIDIA Jetson Orin Nano Super Developer Kit.