Skip to main content

Optimum DeepSparse is an extension of the Hugging Face Transformers library that integrates the DeepSparse inference runtime. DeepSparse offers GPU-class performance on CPUs, making it possible to run Transformers and other deep learning models on commodity hardware with sparsity. Optimum DeepSparse provides a framework for developers to easily integrate DeepSparse into their applications, regardless of the hardware platform.

Project description


Accelerated inference of 🤗 models on CPUs using the DeepSparse Inference Runtime.

DeepSparse Modeling / Python - Test DeepSparse Modeling Nightly


Optimum DeepSparse is a fast-moving project, and you may want to install from source.

pip install git+

Installing in developer mode

If you are working on the optimum-deepsparse code then you should use an editable install by cloning and installing optimum and optimum-deepsparse:

git clone
git clone
pip install -e optimum -e optimum-deepsparse

Now whenever you change the code, you'll be able to run with those changes instantly.

How to use it?

To load a model and run inference with DeepSparse, you can just replace your AutoModelForXxx class with the corresponding DeepSparseModelForXxx class.

import requests
from PIL import Image

- from transformers import AutoModelForImageClassification
+ from optimum.deepsparse import DeepSparseModelForImageClassification
from transformers import AutoFeatureExtractor, pipeline

url = ""
image =, stream=True).raw)

model_id = "microsoft/resnet-50"
- model = AutoModelForImageClassification.from_pretrained(model_id)
+ model = DeepSparseModelForImageClassification.from_pretrained(model_id, export=True, input_shapes="[1,3,224,224]")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
cls_pipe = pipeline("image-classification", model=model, feature_extractor=feature_extractor)
outputs = cls_pipe(image)
Supported Task Model Class
"image-classification" DeepSparseModelForImageClassification
"text-classification"/"sentiment-analysis" DeepSparseModelForSequenceClassification
"audio-classification" DeepSparseModelForAudioClassification
"question-answering" DeepSparseModelForQuestionAnswering
"image-segmentation" DeepSparseModelForSemanticSegmentation

If you find any issue while using those, please open an issue or a pull request.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

optimum-deepsparse-0.1.0.dev1.tar.gz (26.9 kB view hashes)

Uploaded Source

Built Distribution

optimum_deepsparse-0.1.0.dev1-py3-none-any.whl (27.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page