Features

• Easy inference with SAM, supporting bounding boxes, points, masks, and text prompts

• Automatic mask generation

• Finetune SAM on custom datasets

• GroundingDINO integration for text prompt segmentation

• Training a custom decoder to auto segment a specific type of object

• Training a prompt embedding to auto segment a specific type of object

Installation

First, install groundingdino from its repository, this is a dependency for segmate:

NOTE: There is an issue with the setup script in the GroundingDINO repository causing it not able to install torch properly, please manually install PyTorch for now. For other issues, refer to the installation guide:

pip install -U git+https://github.com/IDEA-Research/GroundingDINO.git

Then, install segmate from PyPI:

pip install segmate

Example Usage

To use the provided code snippets, follow the steps below:

1. Import the required modules and initialize the necessary objects:

import torch

from segmate.segmenter import SAM
from segmate.object_detector import GroundingDINO
import segmate.utils as utils

# Model checkpoint path for GroundingDINO is optional. If no path provided, it will download from HuggingFace
od = GroundingDINO(device='cuda', ckpt_path='PATH_TO_CHECKPOINT')
sm = SAM(model_type='MODEL_TYPE', checkpoint='PATH_to_CHECKPOINT', device='cuda')

2. Perform segmentation with bounding box prompts:

masks = sm.segment(image=input_image, boxes_prompt=bbox)
utils.show_masks(image, masks)

Documentation

Detailed package documentation: SegMate Docs

If you have any questions or need assistance, please don’t hesitate to reach out to our support team or join our community forum. We hope you find this toolkit valuable and look forward to seeing the incredible applications you create with SAM!

License

The code in this repository is published under 3-Clause BSD license (see LICENSE file).