onnx-diagnostic 0.2.2

Investigate ONNX models

Helps investigating onnx models, exporting modes into onnx. See documentation of onnx-diagnostic.

Getting started

git clone https://github.com/sdpython/onnx-diagnostic.git
cd onnx-diagnostic
pip install -e .

or

pip install onnx-diagnostic

Enlightening Examples

Torch Export

• Use DYNAMIC or AUTO when exporting if dynamic shapes has constraints

• Find and fix an export issue due to dynamic shapes

• Export with DynamicCache and dynamic shapes

• Steel method forward to guess the dynamic shapes (with Tiny-LLM)

• Export Tiny-LLM with patches

Investigate ONNX models

• Find where a model is failing by running submodels

• Intermediate results with (ONNX) ReferenceEvaluator

• Intermediate results with onnxruntime

Snapshot of usefuls tools

string_type

import torch
from onnx_diagnostic.helpers import string_type

inputs = (
    torch.rand((3, 4), dtype=torch.float16),
    [
        torch.rand((5, 6), dtype=torch.float16),
        torch.rand((5, 6, 7), dtype=torch.float16),
    ]
)

# with shapes
print(string_type(inputs, with_shape=True))

>>> (T10s3x4,#2[T10s5x6,T10s5x6x7])

onnx_dtype_name

import onnx
from onnx_diagnostic.helpers import onnx_dtype_name

itype = onnx.TensorProto.BFLOAT16
print(onnx_dtype_name(itype))
print(onnx_dtype_name(7))

>>> BFLOAT16
>>> INT64

max_diff

Returns the maximum discrancies across nested containers containing tensors.