MLAgility Benchmark and Tools
Project description
The MLAgility Project
MLAgility offers a complementary approach to MLPerf by examining the capability of vendors to provide turnkey solutions to a corpus of hundreds of off-the-shelf models. All of the model scripts and benchmarking code are published as open source software. The performance data is available at our Huggingface Space.
Benchmarking Tool
Our benchit CLI allows you to benchmark Pytorch models without changing a single line of code. The demo below shows BERT-Base being benchmarked on both Nvidia A100 and Intel Xeon. For more information, check out our Tutorials and Tools User Guide.
You can reproduce this demo by trying out the Just Benchmark BERT tutorial.
1000+ Models
This repository is home to a diverse corpus of hundreds of models. We are actively working on increasing the number of models on our model library. You can see the set of models in each category by clicking on the corresponding badge.
Benchmarking results
We are also working on making MLAgility results publicly available at our Huggingface Space. Check it out!
How everything fits together
The diagram above illustrates the MLAgility repository's structure. Simply put, the MLAgility models are leveraged by our benchmarking tool, benchit, to produce benchmarking outcomes showcased on our Hugging Face Spaces page.
Installation
Please refer to our mlagility installation guide to get instructions on how to install mlagility.
Contributing
We are actively seeking collaborators from across the industry. If you would like to contribute to this project, please check out our contribution guide.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mlagility-3.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f066f0c24f24c0cc7c477d61d406d2238942806140db2b0e982c9f3934955a4a |
|
MD5 | 091d9262865d68a10dd317937934651a |
|
BLAKE2b-256 | f295fbdaaef88ce15e63507791a67e673e384ad93b3065b2eb849aab8819f9d2 |