This release is a pre-release and may not be stable for production use.
🚀 What is inference-models?
inference-models is the library to make predictions from computer vision models provided by Roboflow — designed to
be fast, reliable, and user-friendly. It offers:
- Multi-Backend Support: Run models with PyTorch, ONNX, TensorRT, or Hugging Face backends
- Automatic Model Loading: Smart model resolution and backend selection
- Minimal Dependencies: Composable extras system for installing only what you need
- Behavior-Based Interfaces: Models with similar behavior share consistent APIs; custom models can define their own
- Full Roboflow Platform Support: Run any model trained on Roboflow
Visit our documentation for more information.
🛣️ Roadmap
With release 0.19.0, we have reached the first stable release of inference-models and fully integrated
the package to inference - our main inference package, making it selectable backend for running predictions
from models.
We are still making changes to add new features and models. API should be fairly stable already, but the problems may still occur. If you encounter any issues, please report them.
💻 Installation
CPU installation:
uv pip install inference-models
# or with pip
pip install inference-models
inference-models can be installed with CUDA and TensorRT support - see Installation Guide for more options.
🏃➡️ Usage
Pretrained Models
Load and run a pretrained model:
import cv2
import supervision as sv
from inference_models import AutoModel
# Load pretrained model from Roboflow
model = AutoModel.from_pretrained("rfdetr-base")
# Run inference (works with numpy arrays or torch.Tensor)
image = cv2.imread("<path-to-your-image>")
predictions = model(image)
# Use with supervision
annotator = sv.BoxAnnotator()
annotated = annotator.annotate(image, predictions[0].to_supervision())
Your Roboflow Models
Load and run models trained on the Roboflow platform:
import cv2
import supervision as sv
from inference_models import AutoModel
# Load your custom model from Roboflow
model = AutoModel.from_pretrained(
"<your-project>/<version>",
api_key="<your-api-key>" # model access secured with API key
)
# Run inference (works with numpy arrays or torch.Tensor)
image = cv2.imread("<path-to-your-image>")
predictions = model(image)
# Use with supervision
annotator = sv.BoxAnnotator()
annotated = annotator.annotate(image, predictions[0].to_supervision())
🧠 Supported Model Architectures
- RFDetr
- SAM models family
- Vision-Language Models (Florence, PaliGemma, Qwen, SmolVLM, Moondream)
- OCR (DocTR, EasyOCR, TrOCR)
- YOLO
- and many more
For detailed model documentation, see Supported Models.
🔧 Run your local models
Load your own model implementations from a local directory - models with architectures not in the main inference-models package. This is especially valuable for production deployment of custom models.
from inference_models import AutoModel
model = AutoModel.from_pretrained(
"/path/to/my_custom_model",
allow_local_code_packages=True
)
See Load Models from Local Packages for complete details on creating custom model packages.
📄 License
The inference-models package is licensed under Apache 2.0. Individual models may have different licenses - see the Supported Models for details.
Ready to get started? Head to the Quick Overview →
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file inference_models-0.32.0rc4.tar.gz.
File metadata
- Download URL: inference_models-0.32.0rc4.tar.gz
- Upload date:
- Size: 1.8 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b0601d55716dee0e27f1465e46cf99f5c95a2eceee1cc689fbbfe698b957a8ec
|
|
| MD5 |
a6a9513773e69573a27d4db32ebbe0dc
|
|
| BLAKE2b-256 |
08a8ed7b65bfba56af11b3cfa2e32b4335dddc108f2f1a5cc6eae107e4b31709
|
Provenance
The following attestation bundles were made for inference_models-0.32.0rc4.tar.gz:
Publisher:
publish.pypi.inference_exp.yml on roboflow/inference
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
inference_models-0.32.0rc4.tar.gz -
Subject digest:
b0601d55716dee0e27f1465e46cf99f5c95a2eceee1cc689fbbfe698b957a8ec - Sigstore transparency entry: 2212097385
- Sigstore integration time:
-
Permalink:
roboflow/inference@faae2731953159eab3172ecc7484c7fc9094f971 -
Branch / Tag:
refs/heads/peter/cosmos3-generative - Owner: https://github.com/roboflow
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
self-hosted -
Publication workflow:
publish.pypi.inference_exp.yml@faae2731953159eab3172ecc7484c7fc9094f971 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file inference_models-0.32.0rc4-py3-none-any.whl.
File metadata
- Download URL: inference_models-0.32.0rc4-py3-none-any.whl
- Upload date:
- Size: 2.0 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fa3633fd169e9c278bd94f90613d9a256a077c8483daab969a4bf09d80c498cc
|
|
| MD5 |
55e4c639e7e3b8edf58ac4c7b287b7ab
|
|
| BLAKE2b-256 |
2fc8d20241747086ee8b2259d85d2f0f183e65ad97aeced211ccec92e72818ee
|
Provenance
The following attestation bundles were made for inference_models-0.32.0rc4-py3-none-any.whl:
Publisher:
publish.pypi.inference_exp.yml on roboflow/inference
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
inference_models-0.32.0rc4-py3-none-any.whl -
Subject digest:
fa3633fd169e9c278bd94f90613d9a256a077c8483daab969a4bf09d80c498cc - Sigstore transparency entry: 2212097406
- Sigstore integration time:
-
Permalink:
roboflow/inference@faae2731953159eab3172ecc7484c7fc9094f971 -
Branch / Tag:
refs/heads/peter/cosmos3-generative - Owner: https://github.com/roboflow
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
self-hosted -
Publication workflow:
publish.pypi.inference_exp.yml@faae2731953159eab3172ecc7484c7fc9094f971 -
Trigger Event:
workflow_dispatch
-
Statement type:
