autodistill 0.1.0

👋 Hello

autodistill is a Python package to distill large foundational models into smaller, domain-specific models for deployment. Using autodistill, you can go from images to inference on a custom model with little to no labeling in between for a range of use cases.

Distilled models are smaller than general vision models and easier to refine and fine-tune to the situation in which your model will be deployed.

You can also use autodistill to annotate images for use with your existing models.

Installation

To install autodistill, run the following command:

pip install autodistill

git clone https://github.com/roboflow/autodistill
cd autodistill
pip install -e .

🚀 Quickstart

See the Quickstart.ipynb notebook for a quick introduction to autodistill. Below, we have condensed key parts of the notebook for a quick introduction to autodistill.

Create an Ontology

from autodistill_grounding_dino import GroundingDINO
from autodistill_yolov8 import YOLOv8

# define an ontology to map class names to our GroundingDINO prompt
ontology = {
    "ontology_type": "class_descriptions",
    "prompts": [
        {
            "class_name": "dog",
            "prompt_type": "text",
            "prompt_value": "all dogs"
        },
    ]
}

# load the base model and set the ontology
base_model = GroundingDINO(ontology)
base_model.set_ontology(ontology)

# see predictions on an image
detections_list = base_model.view_grounding_dino_prediction("./dog.jpeg")

# label all images in a folder called `context_images`
base_model.label("./context_images", extension=".jpeg")

from autodistill_yolov8 import YOLOv8
target_model = YOLOv8("yolov8n.pt")
target_model.train("./context_images_labeled/data.yaml", epochs=200)

# run inference on the new model
pred = target_model.predict("./context_images_labeled/train/images/dog-7.jpg", conf=0.01)

# optional: upload your model to Roboflow for deployment
from roboflow import Roboflow

rf = Roboflow(api_key="API_KEY")
project = rf.workspace().project("PROJECT_ID")
project.version(DATASET_VERSION).deploy(model_type="yolov8", model_path=f"{HOME}/runs/detect/train/")

base_models are models that have been pre-trained to have a wide breadth of knowledge.

Base models have ontologies which are the prompts we can use to draw predictions from them. Deciding the proper ontology (and installing your CUDA drivers) is most of the work you will have to do to train distilled models with autodistill.

To distill a model, you will need to bring example data of the context you want to your model to operate in.

target_models are smaller in-domain models that you will train using the annotations generated by your base model.

Annotate a Single Image

To plot the annotations for a single image using autodistill, you can use the code below. This code is helpful to visualize the annotations generated by your base model (i.e. Grounding DINO) and the results from your target model (i.e. YOLOv8).

import supervision as sv
import cv2

img_path = "./context_images/dog.jpeg"

image = cv2.imread(img_path)

detections = base_model.predict(img_path)
# annotate image with detections
box_annotator = sv.BoxAnnotator()

labels = [
    f"{base_model.ontology.class_id_map[class_id]} {confidence:0.2f}"
    for _, _, confidence, class_id, _ in detections
]

annotated_frame = box_annotator.annotate(
    scene=image.copy(), detections=detections, labels=labels
)

sv.plot_image(annotated_frame, (16, 16))

📍 roadmap

🚧 - work in progress

object detection

instance segmentation

classification

🏆 Contributing

We love your input! Please see our contributing guide to get started. Thank you 🙏 to all our contributors!

License

The autodistill package is licensed under an MIT license. Each model that integrates with autodistill uses their own license. Please refer to the license associated with each supported model for more information.