A unified interface to run inference on machine learning libraries.
Project description
xinfer
Run computer vision inference with X model framework of your choice.
Why xinfer?
If you'd like to run many models from different libraries without having to rewrite your inference code, xinfer is for you. It has a simple API and is easy to extend. Currently supports Transformers, Ultralytics, and TIMM.
Have a custom machine learning model? Create a class that implements the BaseModel
interface and register it with xinfer. See Adding New Models for more details.
Key Features
- Unified Interface: Interact with different machine learning models through a single, consistent API.
- Modular Design: Integrate and swap out models without altering the core framework.
- Ease of Use: Simplifies model loading, input preprocessing, inference execution, and output postprocessing.
- Extensibility: Add support for new models and libraries with minimal code changes.
Supported Libraries
- Hugging Face Transformers: Natural language processing models for tasks like text classification, translation, and summarization.
- Ultralytics YOLO: State-of-the-art real-time object detection models.
- Custom Models: Support for your own machine learning models and architectures.
Prerequisites
Install PyTorch.
Installation
Install xinfer using pip:
pip install xinfer
With specific libraries:
pip install "xinfer[transformers]"
pip install "xinfer[ultralytics]"
pip install "xinfer[timm]"
Or locally:
pip install -e .
With specific libraries:
pip install -e ."[transformers]"
pip install -e ."[ultralytics]"
pip install -e ."[timm]"
Getting Started
Here's a quick example demonstrating how to use xinfer with a Transformers model:
import xinfer
# Instantiate a Transformers model
model = xinfer.create_model("vikhyatk/moondream2")
# Input data
image = "https://raw.githubusercontent.com/vikhyat/moondream/main/assets/demo-1.jpg"
prompt = "Describe this image. "
# Run inference
output = model.inference(image, prompt, max_new_tokens=50)
print(output)
>>> An animated character with long hair and a serious expression is eating a large burger at a table, with other characters in the background.
See example.ipynb for more examples.
Supported Models
Transformers:
- BLIP2 Series
model = xinfer.create_model("Salesforce/blip2-opt-2.7b")
- Moondream2
model = xinfer.create_model("vikhyatk/moondream2")
[!NOTE] Wish to load an unlisted model? You can load any Vision2Seq model from Transformers by using the
Vision2SeqModel
class.
from xinfer.transformers import Vision2SeqModel
model = Vision2SeqModel("facebook/chameleon-7b")
model = xinfer.create_model(model)
TIMM:
- EVA02 Series
model = xinfer.create_model("eva02_small_patch14_336.mim_in22k_ft_in1k")
[!NOTE] Wish to load an unlisted model? You can load any model from TIMM by using the
TIMMModel
class.
from xinfer.timm import TimmModel
model = TimmModel("resnet18")
model = xinfer.create_model(model)
Ultralytics:
- YOLOv8 Series
model = xinfer.create_model("yolov8n")
- YOLOv10 Series
model = xinfer.create_model("yolov10x")
- YOLOv11 Series
model = xinfer.create_model("yolov11s")
[!NOTE] Wish to load an unlisted model? You can load any model from Ultralytics by using the
UltralyticsModel
class.
from xinfer.ultralytics import UltralyticsModel
model = UltralyticsModel("yolov5n6u")
model = xinfer.create_model(model)
Get a list of available models:
import xinfer
xinfer.list_models()
Available Models | ||
---|---|---|
Implementation | Model ID | Input --> Output |
timm | eva02_large_patch14_448.mim_m38m_ft_in22k_in1k | image --> class |
timm | eva02_large_patch14_448.mim_m38m_ft_in1k | image --> class |
timm | eva02_large_patch14_448.mim_in22k_ft_in22k_in1k | image --> class |
timm | eva02_large_patch14_448.mim_in22k_ft_in1k | image --> class |
timm | eva02_base_patch14_448.mim_in22k_ft_in22k_in1k | image --> class |
timm | eva02_base_patch14_448.mim_in22k_ft_in1k | image --> class |
timm | eva02_small_patch14_336.mim_in22k_ft_in1k | image --> class |
timm | eva02_tiny_patch14_336.mim_in22k_ft_in1k | image --> class |
transformers | Salesforce/blip2-opt-6.7b-coco | image-text --> text |
transformers | Salesforce/blip2-flan-t5-xxl | image-text --> text |
transformers | Salesforce/blip2-opt-6.7b | image-text --> text |
transformers | Salesforce/blip2-opt-2.7b | image-text --> text |
transformers | vikhyatk/moondream2 | image-text --> text |
ultralytics | yolov8x | image --> objects |
ultralytics | yolov8m | image --> objects |
ultralytics | yolov8l | image --> objects |
ultralytics | yolov8s | image --> objects |
ultralytics | yolov8n | image --> objects |
ultralytics | yolov10x | image --> objects |
ultralytics | yolov10m | image --> objects |
... | ||
... |
Adding New Models
-
Step 1: Create a new model class that implements the
BaseModel
interface. -
Step 2: Implement the required abstract methods
load_model
andinference
. -
Step 3: Decorate your class with the
register_model
decorator, specifying the model ID, backend, and input/output.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for xinfer-0.0.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69576d050c49374961d3832e975029c846c8007c26fd2a55bf387546f22ec39b |
|
MD5 | dfd03e56c09b7fa83c0b74545c581b04 |
|
BLAKE2b-256 | 84f49f426a17d48e50ac5af2e28d8628e7dc3a2d96e61084bf61edd13dc90a88 |