Skip to main content

RobustSAM: Segment Anything Robustly on Degraded Images.

Project description

RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)

Official repository for RobustSAM: Segment Anything Robustly on Degraded Images

Project Page | Paper | Video | Dataset

Updates

  • June 2024: ✨ Inference code of RobustSAM was released!
  • Feb 2024: ✨ RobustSAM was accepted into CVPR 2024!

Introduction

Segment Anything Model (SAM) has emerged as a transformative approach in image segmentation, acclaimed for its robust zero-shot segmentation capabilities and flexible prompting system. Nonetheless, its performance is challenged by images with degraded quality. Addressing this limitation, we propose the Robust Segment Anything Model (RobustSAM), which enhances SAM's performance on low-quality images while preserving its promptability and zero-shot generalization.

Our method leverages the pre-trained SAM model with only marginal parameter increments and computational requirements. The additional parameters of RobustSAM can be optimized within 30 hours on eight GPUs, demonstrating its feasibility and practicality for typical research laboratories. We also introduce the Robust-Seg dataset, a collection of 688K image-mask pairs with different degradations designed to train and evaluate our model optimally. Extensive experiments across various segmentation tasks and datasets confirm RobustSAM's superior performance, especially under zero-shot conditions, underscoring its potential for extensive real-world application. Additionally, our method has been shown to effectively improve the performance of SAM-based downstream tasks such as single image dehazing and deblurring.

image

Setup

  1. Create a conda environment and activate it.
conda create --name robustsam python=3.10 -y
conda activate robustsam
  1. Clone and enter into repo directory.
git clone https://github.com/robustsam/RobustSAM
cd RobustSAM
  1. Use command below to check your CUDA version.
nvidia-smi
  1. Replace the CUDA version with yours in command below.
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu[$YOUR_CUDA_VERSION]
# For example: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 # cu117 = CUDA_version_11.7
  1. Install remaining dependencies
pip install -r requirements.txt
  1. Download pretrained RobustSAM checkpoint and place into current directory.

Demo

We have prepared some images im demo_images folder for demo purpose. Besides, two prompting modes are available (box prompts and point prompts).

  • For box prompt:
python eval.py --bbox
  • For point prompt:
python eval.py

In default, demo results will be saved to demo_result/[$PROMPT_TYPE].

Comparison of computational requirements

image

Visual Comparison

image

Quantitative Comparison

Seen dataset with synthetic degradation

image

Unseen dataset with synthetic degradation

image

Unseen dataset with real degradation

image

Reference

If you find this work useful, please consider citing us!

@inproceedings{chen2024robustsam,
  title={RobustSAM: Segment Anything Robustly on Degraded Images},
  author={Chen, Wei-Ting and Vong, Yu-Jiet and Kuo, Sy-Yen and Ma, Sizhou and Wang, Jian},
  journal={CVPR},
  year={2024}
}

Acknowledgements

We thank the authors of SAM from which our repo is based off of.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

robust_segment_anything-1.0.0.tar.gz (33.5 kB view details)

Uploaded Source

Built Distribution

robust_segment_anything-1.0.0-py3-none-any.whl (39.7 kB view details)

Uploaded Python 3

File details

Details for the file robust_segment_anything-1.0.0.tar.gz.

File metadata

File hashes

Hashes for robust_segment_anything-1.0.0.tar.gz
Algorithm Hash digest
SHA256 a6064227bf4ff780f6a4bb815a8f508802c085fe46fd960a86ada8f2f7db9940
MD5 72bf5ba21a463cf039154c550fe48b07
BLAKE2b-256 185560b6d2f424fee8a80af41031a76c9b7a994a072a9e217b44bf322e1f35d3

See more details on using hashes here.

File details

Details for the file robust_segment_anything-1.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for robust_segment_anything-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e4ad924578ee197e438a37535416d3a3c8dbab1d9017241f6ce12ea8433080ca
MD5 eda0d62dcf7c16a496b5930363ef492f
BLAKE2b-256 e33e5482f5176f64298e2baeb3fbfbdedb52c2f78136f5c177512475b1b8b811

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page