NNSmith

🌟NNSmith🌟 is a random DNN generator and a fuzzing infrastructure, primarily designed for automatically validating deep-learning frameworks and compilers.

Support Table

Models	`tvm`	`onnxruntime`	`tensorrt`	`tflite`	`xla`	`torchjit`
ONNX	✅	✅	✅
TensorFlow	🔨			✅	✅
PyTorch	🔨	🔨				✅

✅: Supported; 🔨: Coming soon;

Quick Start

Install latest code (GitHub HEAD):

pip install "git+https://github.com/ise-uiuc/nnsmith@main#egg=nnsmith[torch,onnx]" --upgrade
# [optional] add more front- and back-ends such as [tensorflow] and [tvm,onnxruntime,xla,...] in "[...]"

Install latest stable release [click]

pip install "nnsmith[torch,onnx]" --upgrade

Install latest pre-release [click]

pip install "nnsmith[torch,onnx]" --upgrade --pre

Setting up graphviz for debugging [click]

Graphviz provides dot for visualizing graphs in nice pictures. But it needs to be installed via the following methods:

sudo apt-get install graphviz graphviz-dev      # Linux
brew install graphviz                           # MacOS
conda install --channel conda-forge pygraphviz  # Conda
choco install graphviz                          # Windows

pip install pygraphviz  # Final step.

Also see pygraphviz install guidance.

# Generate a random model in "nnsmith_outputs/*"
nnsmith.model_gen model.type=onnx debug.viz=true

Learning More

Bugs: links to reports.
Documentation: CLI, concept, logging, and known issues.
Contributions: doc/CONTRIBUTING.md
We use hydra to manage configurations. See nnsmith/config/main.yaml.

Papers

📜 NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers. [click :: citation]

@inproceedings{liu2023nnsmith,
  title={Nnsmith: Generating diverse and valid test cases for deep learning compilers},
  author={Liu, Jiawei and Lin, Jinkun and Ruffy, Fabian and Tan, Cheng and Li, Jinyang and Panda, Aurojit and Zhang, Lingming},
  booktitle={Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
  pages={530--543},
  year={2023}
}

Release files for nnsmith 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nnsmith 0.1.0
File	Size	Uploaded
nnsmith-0.1.0.tar.gz	131.4 kB	Mar 17, 2023	Details

Built distribution (wheel)

Table of built distributions (wheels) for nnsmith 0.1.0
File	Interpreter	ABI	Platform	Reset
nnsmith-0.1.0-py3-none-any.whl 90.5 kB Mar 17, 2023	Python 3	none	any	Details

Total release size: 221.9 kB

Release files / nnsmith-0.1.0.tar.gz

Download URL	nnsmith-0.1.0.tar.gz
Size	131.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`606f463b66d8f8fe0cce8103885204b64ed0689c6867ad83df76bbd138d4c433`
BLAKE2b-256 checksum How to use checksums	`bd0a8bbb6c4c26f30eefbb25c11699cf9b3f6a784088f2ed67b969d872745182`
Upload date	Mar 17, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.2 CPython/3.8.13`

Release files / nnsmith-0.1.0-py3-none-any.whl

Download URL	nnsmith-0.1.0-py3-none-any.whl
Size	90.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`251d5c0cc9b7f77ed336006c8818e9879372e8afff64c620b0b595a569e40084`
BLAKE2b-256 checksum How to use checksums	`de49e6c8452992e4a1d530eef4034366b262cae969c311450da5ad419b3956a0`
Upload date	Mar 17, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.2 CPython/3.8.13`