Green AI Profiler

Open-source AI model profiler for PyTorch (and ONNX) that measures latency, throughput, memory, energy consumption, and layer-wise bottlenecks—so you can build faster, cheaper, and greener models without guessing.

Search terms this project covers: AI model profiler, PyTorch latency benchmark, ML energy measurement, green AI, model efficiency toolkit, inference performance profiler, NVML / RAPL energy, layer-wise profiling, sustainable machine learning.

pip install "green-ai-profiler[torch]"
greenai profile model.pt --allow-pickle --input-shape 1,28,28

What you can measure

Why this exists

Accuracy alone is not enough. ML engineers, researchers, and MLOps teams also need:

• model latency and samples/sec
• memory and peak RSS behavior
• parameter count and model size
• energy per inference and carbon-aware efficiency (when measurable)

Energy is hardware-dependent. This project never fabricates joules. If a backend is unavailable, you get an explicit status and reason.

How people run this

1. Install from PyPI (recommended)

Requirements: Python 3.11+

python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

pip install -U pip
pip install "green-ai-profiler[torch]"

Useful extras:

Verify the install:

greenai --version
greenai --help
greenai system-info

From source (contributors)

git clone https://github.com/Md-Sifat-Bin-Jibon/greenai-profiler.git
cd greenai-profiler
python -m venv .venv
# activate the venv, then:
pip install -e ".[torch,dev]"

2. Profile a model (CLI)

Try the built-in example first:

python examples/save_example_model.py

greenai inspect examples/artifacts/tiny_cnn.pt --allow-pickle --input-shape 1,28,28
greenai benchmark examples/artifacts/tiny_cnn.pt --allow-pickle --input-shape 1,28,28
greenai profile examples/artifacts/tiny_cnn.pt --allow-pickle --input-shape 1,28,28 --output results.json
greenai report results.json --html report.html

Then point greenai at your own checkpoint:

# CPU
greenai profile path/to/model.pt --allow-pickle --input-shape 3,224,224

# CUDA (if PyTorch CUDA is available)
greenai profile path/to/model.pt --allow-pickle --device cuda --input-shape 3,224,224

# Save JSON for later comparison / HTML report
greenai profile path/to/model.pt --allow-pickle --input-shape 3,224,224 --output results.json
greenai compare baseline.json optimized.json
greenai report results.json --html report.html

--allow-pickle is required for full torch.save(model, ...) module checkpoints. Only use it for models you trust. Prefer a state_dict plus reconstructed architecture — see examples/state_dict_workflow.py and docs/security.md.

3. Use it from Python

import torch
from greenai.benchmark import BenchmarkConfig, run_benchmark
from greenai.models.pytorch import PyTorchModelAdapter

model = torch.nn.Sequential(
    torch.nn.Flatten(),
    torch.nn.Linear(64, 32),
    torch.nn.ReLU(),
    torch.nn.Linear(32, 4),
)
adapter = PyTorchModelAdapter(model, device="cpu", example_input_shape=[64])
result = run_benchmark(
    adapter,
    BenchmarkConfig(device="cpu", input_shape=[64], warmup=2, iterations=10),
)

print(f"mean latency: {result.latency.mean_seconds * 1000:.3f} ms")
print(f"energy status: {result.energy.status if result.energy else 'n/a'}")

Runnable copy: examples/basic_benchmark.py.

4. CLI reference

Common options on profile/benchmark commands:

• --device cpu|cuda
• --batch-size N
• --input-shape C,H,W (no batch dim)
• --warmup N / --iterations N
• --allow-pickle
• --output path / --format json|csv
• --no-energy

Example output

╭──────── Green AI Profiler ────────╮
Model
  TinyCNN

Performance
  Latency:       0.42 ms
  P95:           0.55 ms
  Throughput:    2380.1 samples/sec

Memory
  Peak/RSS:      184.12 MB

Energy
  Status:        unavailable
  Reason:        No supported hardware energy interface detected.

Model Stats
  Parameters:    11.98K
  Size:          47.92 KB
╰───────────────────────────────────╯

Supported hardware

Details: docs/hardware-support.md, docs/energy-measurement.md.

Architecture

CLI (Typer)
   │
   ▼
Models ──► Benchmark runner ──► Reporting (terminal / JSON / CSV / HTML)
   │              │
   │              ├── Latency / throughput
   │              ├── Memory (CPU RSS, CUDA)
   │              └── Energy monitors (NVML / RAPL / unavailable)
   │
   └── Layer profiler ──► Bottlenecks ──► Recommendations

See docs/architecture.md.

Documentation

Limitations

• Pickle-based .pt files can execute code; safe loading is default, --allow-pickle is explicit.
• Process RSS is not model-only memory.
• Energy requires real hardware telemetry; estimates are labeled if ever used.
• ONNX support is inspection-oriented in this release.
• Green Score is a project-defined heuristic, not a scientific standard.

Development

pip install -e ".[dev,torch]"
ruff check src tests
mypy src
pytest

Roadmap

• Broader ONNX Runtime benchmarking
• Static INT8 calibration workflows
• Edge/Android measurement import protocol
• Richer visualization pack
• Expanded hardware energy backends

License

Apache License 2.0 — see LICENSE.