hidet 0.2.3

Hidet: a compilation-based DNN inference framework.

Hidet: An open-source deep learning framework

Documentation | Research Paper | Releases | Contributing

Hidet is an open-source deep learning inference framework based on compilation. It supports end-to-end compilation of DNN models from PyTorch and ONNX to efficient cuda kernels. A series of graph-level and operator-level optimizations are applied to optimize the performance.

Getting Started

Installation

pip install hidet

See here for building from source.

Usage

Optimize a PyTorch model through hidet (require PyTorch 2.0):

import torch

# Define pytorch model
model = torch.hub.load('pytorch/vision:v0.6.0', 'resnet18', pretrained=True).cuda().eval()
x = torch.rand(1, 3, 224, 224).cuda()

# Compile the model through Hidet
# Optional: set optimization options (see our documentation for more details)
#   import hidet 
#   hidet.torch.dynamo_config.search_space(2)  # tune each tunable operator
#   hidet.torch.dynamo_config.use_fp16()       # use float16 for acceleration
model_opt = torch.compile(model, backend='hidet')  

# Run the optimized model
y = model_opt(x)

See the following tutorials to learn other usages:

• Quick Start
• Optimize PyTorch models
• Optimize ONNX models

Publication

Hidet originates from the following research work.

Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs
Yaoyao Ding, Cody Hao Yu, Bojian Zheng, Yizhi Liu, Yida Wang, and Gennady Pekhimenko.
To appear in ASPLOS 2023.

If you used Hidet in your research, welcome to cite our paper.

Development

Hidet is currently under active development by a team at CentML Inc.

Contributing

We welcome contributions from the community. Please see contribution guide for more details.

License

Hidet is released under the Apache 2.0 license.