Skip to main content

Auto-label data with a PaliGemma model, or ine-tune a PaLiGemma model using custom data with Autodistill.

Project description

Autodistill PaLiGemma Module

This repository contains the code supporting the PaLiGemma base model for use with Autodistill.

PaLiGemma, developed by Google, is a computer vision model trained using pairs of images and text. You can label data with PaliGemma models for use in training smaller, fine-tuned models with Autodisitll.

Read the full Autodistill documentation.

Installation

To use PaLiGemma with autodistill, you need to install the following dependency:

pip3 install autodistill-paligemma

Quickstart

Auto-label with an existing model

from autodistill_paligemma import PaliGemma

# define an ontology to map class names to our PaliGemma prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated annotations
# then, load the model
base_model = PaliGemma(
    ontology=CaptionOntology(
        {
            "person": "person",
            "a forklift": "forklift"
        }
    )
)

# label a single image
result = PaliGemma.predict("test.jpeg")
print(result)

# label a folder of images
base_model.label("./context_images", extension=".jpeg")

Model fine-tuning

You can fine-tune PaliGemma models with LoRA for deployment with Roboflow Inference.

To train a model, use this code:

from autodistill_paligemma import PaLiGemmaTrainer

target_model = PaLiGemmaTrainer()

# train a model
target_model.train("./data/")

License

The model weights for PaLiGemma are licensed under a custom Google license. To learn more, refer to the Google Gemma Terms of Use.

🏆 Contributing

We love your input! Please see the core Autodistill contributing guide to get started. Thank you 🙏 to all our contributors!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

autodistill_paligemma-0.1.1.tar.gz (4.9 kB view details)

Uploaded Source

Built Distribution

autodistill_paligemma-0.1.1-py3-none-any.whl (5.0 kB view details)

Uploaded Python 3

File details

Details for the file autodistill_paligemma-0.1.1.tar.gz.

File metadata

  • Download URL: autodistill_paligemma-0.1.1.tar.gz
  • Upload date:
  • Size: 4.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.7

File hashes

Hashes for autodistill_paligemma-0.1.1.tar.gz
Algorithm Hash digest
SHA256 b0d7edace5ebc30587c57c24658e42815dcbac3cd84fd98338315476a35b93d0
MD5 dc85b885344718a052c1e200ee54de5f
BLAKE2b-256 12fef5a7b0c40c96089716566c7d2f73075202ba67bf43458ba62576028de0bb

See more details on using hashes here.

File details

Details for the file autodistill_paligemma-0.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for autodistill_paligemma-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 c3695250687bfae317789a9c572b56d0742556869ff643b019ef9f01eb77bcdc
MD5 6e1598364ff9346a334ae187dbaf6ba2
BLAKE2b-256 b8b437333cab15564caf460a34faff49cde4db5875a5306765de084c1befb56a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page