autodistill 0.1.1

Distill large foundational models into smaller, domain-specific models for deployment

👋 Hello

autodistill is a Python package to distill large foundational models into smaller, domain-specific models for deployment. Using autodistill, you can go from images to inference on a custom model with little to no labeling in between for a range of use cases.

Distilled models are smaller than general vision models and easier to refine and fine-tune to the situation in which your model will be deployed.

You can also use autodistill to annotate images for use with your existing models.

Installation

To install autodistill, run the following command:

pip install autodistill

Install from source

You can also clone the project from GitHub for local development:

git clone https://github.com/roboflow/autodistill
cd autodistill
pip install -e .

🚀 Quickstart

See the Quickstart.ipynb notebook for a quick introduction to autodistill. Below, we have condensed key parts of the notebook for a quick introduction to autodistill.

Create an Ontology

from autodistill_grounding_dino import GroundingDINO
from autodistill_yolov8 import YOLOv8

# define an ontology to map class names to our GroundingDINO prompt
ontology = {
    "ontology_type": "class_descriptions",
    "prompts": [
        {
            "class_name": "dog",
            "prompt_type": "text",
            "prompt_value": "all dogs"
        },
    ]
}

# load the base model and set the ontology
base_model = GroundingDINO(ontology)
base_model.set_ontology(ontology)

# see predictions on an image
detections_list = base_model.view_grounding_dino_prediction("./dog.jpeg")

# label all images in a folder called `context_images`
base_model.label("./context_images", extension=".jpeg")

from autodistill_yolov8 import YOLOv8
target_model = YOLOv8("yolov8n.pt")
target_model.train("./context_images_labeled/data.yaml", epochs=200)

# run inference on the new model
pred = target_model.predict("./context_images_labeled/train/images/dog-7.jpg", conf=0.01)

# optional: upload your model to Roboflow for deployment
from roboflow import Roboflow

rf = Roboflow(api_key="API_KEY")
project = rf.workspace().project("PROJECT_ID")
project.version(DATASET_VERSION).deploy(model_type="yolov8", model_path=f"{HOME}/runs/detect/train/")

base_models are models that have been pre-trained to have a wide breadth of knowledge.

Base models have ontologies which are the prompts we can use to draw predictions from them. Deciding the proper ontology (and installing your CUDA drivers) is most of the work you will have to do to train distilled models with autodistill.

To distill a model, you will need to bring example data of the context you want to your model to operate in.

target_models are smaller in-domain models that you will train using the annotations generated by your base model.

Annotate a Single Image

To plot the annotations for a single image using autodistill, you can use the code below. This code is helpful to visualize the annotations generated by your base model (i.e. Grounding DINO) and the results from your target model (i.e. YOLOv8).

import supervision as sv
import cv2

img_path = "./context_images/dog.jpeg"

image = cv2.imread(img_path)

detections = base_model.predict(img_path)
# annotate image with detections
box_annotator = sv.BoxAnnotator()

labels = [
    f"{base_model.ontology.class_id_map[class_id]} {confidence:0.2f}"
    for _, _, confidence, class_id, _ in detections
]

annotated_frame = box_annotator.annotate(
    scene=image.copy(), detections=detections, labels=labels
)

sv.plot_image(annotated_frame, (16, 16))

📍 roadmap

🚧 - work in progress

object detection

base / target	YOLOv5	YOLOv7	YOLOv8	RT-DETR
Grounded DINO
Grounded SAM			🚧
DETIC
OWL-ViT

instance segmentation

base / target	YOLOv5	YOLOv7	YOLOv8	RT-DETR
Grounded SAM			🚧

classification

base / target	YOLOv8	YOLOv5
CLIP

🏆 Contributing

We love your input! Please see our contributing guide to get started. Thank you 🙏 to all our contributors!

License

The autodistill package is licensed under an MIT license. Each model that integrates with autodistill uses their own license. Please refer to the license associated with each supported model for more information.