ONNX Runtime
ONNX Runtime enables high-performance evaluation of trained machine learning (ML) models while keeping resource usage low. Building on Microsoft’s dedication to the Open Neural Network Exchange (ONNX) community, it supports traditional ML models as well as Deep Learning algorithms in the ONNX-ML format. Documentation is available at Python Bindings for ONNX Runtime.
Example
The following example demonstrates an end-to-end example in a very common scenario. A model is trained with scikit-learn but it has to run very fast in a optimized environment. The model is then converted into ONNX format and ONNX Runtime replaces scikit-learn to compute the predictions.
# Train a model.
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y)
clr = RandomForestClassifier()
clr.fit(X_train, y_train)
# Convert into ONNX format with onnxmltools
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
initial_type = [('float_input', FloatTensorType([1, 4]))]
onx = convert_sklearn(clr, initial_types=initial_type)
with open("rf_iris.onnx", "wb") as f:
f.write(onx.SerializeToString())
# Compute the prediction with ONNX Runtime
import onnxruntime as rt
import numpy
sess = rt.InferenceSession("rf_iris.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]
Changes
0.4.0
Release Notes : https://github.com/Microsoft/onnxruntime/releases/tag/v0.4.0
0.3.1
Protobuf-lite, NuGet file fixes (patch to 0.3.0).
0.3.0
C-API, Linux support for Dotnet Nuget package, Cuda 9.1 support.
0.2.1
C-API, Linux support for Dotnet Nuget package, Cuda 10.0 support (patch to 0.2.0).
0.2.0
C-API, Linux support for Dotnet Nuget package, Cuda 10.0 support
0.1.5
GA release as part of open sourcing onnxruntime (patch to 0.1.4).
0.1.4
GA release as part of open sourcing onnxruntime.
0.1.3
Fixes a crash on machines which do not support AVX instructions.
0.1.2
First release on Ubuntu 16.04 for CPU and GPU with Cuda 9.1 and Cudnn 7.0, supports runtime for deep learning models architecture such as AlexNet, ResNet, XCeption, VGG, Inception, DenseNet, standard linear learner, standard ensemble learners, and transform scaler, imputer.
1 maintainer
Release files for onnxruntime 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| onnxruntime-0.4.0-cp37-cp37m-win_amd64.whl | CPython 3.7 | CPython 3.7 pymalloc | Windows x86-64 | Details |
| onnxruntime-0.4.0-cp37-cp37m-manylinux1_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | Linux glibc 2.5+ x86-64 | Details |
| onnxruntime-0.4.0-cp37-cp37m-macosx_10_7_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | macOS 10.7+ x86-64 | Details |
| onnxruntime-0.4.0-cp36-cp36m-win_amd64.whl | CPython 3.6 | CPython 3.6 pymalloc | Windows x86-64 | Details |
| onnxruntime-0.4.0-cp36-cp36m-manylinux1_x86_64.whl | CPython 3.6 | CPython 3.6 pymalloc | Linux glibc 2.5+ x86-64 | Details |
| onnxruntime-0.4.0-cp36-cp36m-macosx_10_7_x86_64.whl | CPython 3.6 | CPython 3.6 pymalloc | macOS 10.7+ x86-64 | Details |
| onnxruntime-0.4.0-cp35-cp35m-win_amd64.whl | CPython 3.5 | CPython 3.5 pymalloc | Windows x86-64 | Details |
| onnxruntime-0.4.0-cp35-cp35m-manylinux1_x86_64.whl | CPython 3.5 | CPython 3.5 pymalloc | Linux glibc 2.5+ x86-64 | Details |
| onnxruntime-0.4.0-cp35-cp35m-macosx_10_6_x86_64.whl | CPython 3.5 | CPython 3.5 pymalloc | macOS 10.6+ x86-64 | Details |
Total release size: 22.5 MB
Release files / onnxruntime-0.4.0-cp37-cp37m-win_amd64.whl
| Download URL | onnxruntime-0.4.0-cp37-cp37m-win_amd64.whl |
|---|---|
| Size | 1.9 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
ddcbde5acf3e4720cf9a870c3fec897ce06bacf805bc9406b96cf3d9bebfeb56
|
|
BLAKE2b-256 checksum How to use checksums |
a12b222209ff75a70e56114b927b1e970a8de77d36d932f2eb453ae55d7db173
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp37-cp37m-manylinux1_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp37-cp37m-manylinux1_x86_64.whl |
|---|---|
| Size | 3.1 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
9f94d2754814d436fd948d3b1771d27e75bc8e0b29398df51e588c0a53accd59
|
|
BLAKE2b-256 checksum How to use checksums |
5f6912e7048508193cb9cebca27202c247cc847e75ab7894a21513debdb2d935
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp37-cp37m-macosx_10_7_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp37-cp37m-macosx_10_7_x86_64.whl |
|---|---|
| Size | 2.5 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc macOS 10.7+ x86-64 |
|
SHA-256 checksum How to use checksums |
af38d0c4557eb9e7fef2d239238c16f02533721820744450150198fd6e06bb7d
|
|
BLAKE2b-256 checksum How to use checksums |
a55f103ce0a167892c3924842079b05acc63a6697b4823e261ad6ebf759448e5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp36-cp36m-win_amd64.whl
| Download URL | onnxruntime-0.4.0-cp36-cp36m-win_amd64.whl |
|---|---|
| Size | 1.9 MB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
7927adf4f3e0299a059037d907afbdba7da579e595901b478364b697adec07a5
|
|
BLAKE2b-256 checksum How to use checksums |
af2b003820bfc750b8e11bff1d6842a0d4303d24f31b544304e2874a55742cd8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp36-cp36m-manylinux1_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp36-cp36m-manylinux1_x86_64.whl |
|---|---|
| Size | 3.1 MB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
8cc015c27731f61257661a8f9eff83ed63e7df8e20df9d3abb5126140396f8bc
|
|
BLAKE2b-256 checksum How to use checksums |
fae11548ef61a5c3b583c7ce777c9f38e30b7fdeda309ddcc886b5883cba5771
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp36-cp36m-macosx_10_7_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp36-cp36m-macosx_10_7_x86_64.whl |
|---|---|
| Size | 2.5 MB |
| Tags | CPython 3.6 CPython 3.6 pymalloc macOS 10.7+ x86-64 |
|
SHA-256 checksum How to use checksums |
01e961e558aa9fc678cd2f2046a901f9a8ce392d7f9286d0adb68c1e43cf284b
|
|
BLAKE2b-256 checksum How to use checksums |
0188d0b731180d958e7505d146fad23927df5c87997017ea3fb0def4f62ee187
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp35-cp35m-win_amd64.whl
| Download URL | onnxruntime-0.4.0-cp35-cp35m-win_amd64.whl |
|---|---|
| Size | 1.9 MB |
| Tags | CPython 3.5 CPython 3.5 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
6344d54d66badb9d9c7dc767b416faf2e32ab12e0779a68ced08e24ef726be3c
|
|
BLAKE2b-256 checksum How to use checksums |
4363dbba09de81d1756b116b220f4183309c67234a32b7ff28d46936681d90e9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp35-cp35m-manylinux1_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp35-cp35m-manylinux1_x86_64.whl |
|---|---|
| Size | 3.1 MB |
| Tags | CPython 3.5 CPython 3.5 pymalloc Linux glibc 2.5+ x86-64 |
|
SHA-256 checksum How to use checksums |
afb57e7e43c5466208e8eb656d740eef85a98ccbba22831582a365a87336f25c
|
|
BLAKE2b-256 checksum How to use checksums |
2418576d62d4c87b7d4f74872b4e48261017e0477211e7647080c31a9f8cd8cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
Release files / onnxruntime-0.4.0-cp35-cp35m-macosx_10_6_x86_64.whl
| Download URL | onnxruntime-0.4.0-cp35-cp35m-macosx_10_6_x86_64.whl |
|---|---|
| Size | 2.5 MB |
| Tags | CPython 3.5 CPython 3.5 pymalloc macOS 10.6+ x86-64 |
|
SHA-256 checksum How to use checksums |
ddd311747060217e3d8b4a3c590fd1a25452d66e2e9f3894f3510ac26872ca6a
|
|
BLAKE2b-256 checksum How to use checksums |
2039af77ab5328b68977804d68ffd390e032835daf2a1051b4cc749b5a32d485
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5
|
