nnsmith 0.1.0rc2

"Automatic DNN generation for fuzzing and more."

NNSmith

🌟NNSmith🌟 is a random DNN generator and a fuzzing infrastructure, primarily designed for automatically validating deep-learning frameworks and compilers.

Support Table

Models	tvm	onnxruntime	tensorrt	tflite	xla	torchjit
ONNX	✅	✅	✅
TensorFlow	🔨			✅	✅
PyTorch	🔨	🔨				✅

✅: Supported; 🔨: Coming soon;

Quick Start

Install latest code (GitHub HEAD):

pip install "git+https://github.com/ise-uiuc/nnsmith@main#egg=nnsmith[torch,onnx]" --upgrade
# [optional] add more front- and back-ends such as [tensorflow] and [tvm,onnxruntime,xla,...] in "[...]"

Install latest stable release [click]

pip install "nnsmith[torch,onnx]" --upgrade

Install latest pre-release [click]

pip install "nnsmith[torch,onnx]" --upgrade --pre

Setting up graphviz for debugging [click]

Graphviz provides dot for visualizing graphs in nice pictures. But it needs to be installed via the following methods:

sudo apt-get install graphviz graphviz-dev      # Linux
brew install graphviz                           # MacOS
conda install --channel conda-forge pygraphviz  # Conda
choco install graphviz                          # Windows

pip install pygraphviz  # Final step.

Also see pygraphviz install guidance.

# Generate a random model in "nnsmith_outputs/*"
nnsmith.model_gen model.type=onnx debug.viz=true

Learning More

• Bugs: links to reports.
• Documentation: CLI, concept, logging, and known issues.
• Contributions: doc/CONTRIBUTING.md
• We use hydra to manage configurations. See nnsmith/config/main.yaml.

Papers

📜 NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers. [click :: citation]

@inproceedings{liu2023nnsmith,
  title={Nnsmith: Generating diverse and valid test cases for deep learning compilers},
  author={Liu, Jiawei and Lin, Jinkun and Ruffy, Fabian and Tan, Cheng and Li, Jinyang and Panda, Aurojit and Zhang, Lingming},
  booktitle={Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
  pages={530--543},
  year={2023}
}