coralnet-toolbox 0.0.71

Tools for annotating and developing ML models for benthic imagery

CoralNet-Toolbox 🪸🧰

🔍 Annotation Create patches, rectangles, and polygons with AI assistance

🧠 AI-Powered Leverage SAM, YOLOE, and various foundation models

🚀 Complete Workflow From data collection to model training and deployment

🚦 Quick Start

Running the following command will install the coralnet-toolbox, which you can then run from the command line:

# cmd

# Install
pip install coralnet-toolbox

# Run
coralnet-toolbox

📚 Guides

For further information and instructions, please see the following guides:

• Overview
• Installation
• Usage
• Community Provided:
  • Patch-based Image Classifier

🎥 Watch the Video Demos

⏩ TL;Dr

The CoralNet-Toolbox is an unofficial codebase that can be used to augment processes associated with those on CoralNet.

It uses✨Ultralytics🚀 as a base, which is an open-source library for computer vision and deep learning built in PyTorch. For more information on their AGPL-3.0 license, see here.

🚀 Supported Models

The toolbox integrates a variety of state-of-the-art models to help you create rectangle and polygon annotations efficiently. Below is a categorized overview of the supported models and frameworks:

Category	Models
Trainable	- 🦾 YOLOv3 - 🦈 YOLOv4 - 🦅 YOLOv5 - 🐬 YOLOv6 - 🐢 YOLOv7 - 🐙 YOLOv8 - 🐠 YOLOv9 - 🦑 YOLOv10 - 🚀 YOLO11 - 🐳 YOLO12
Segment Anything	- 🪸 SAM - 🌊 CoralSCOP - ⚡ FastSAM - 🔁 RepViT-SAM - ✂️ EdgeSAM - 📱 MobileSAM
Visual Prompting	- 👁️ YOLOE - 🤖 AutoDistill:    • 🦒 Grounding DINO    • 🦉 OWLViT    • ⚡ OmDetTurbo

These models enable fast, accurate, and flexible annotation workflows for a wide range of use cases for patch-based image classification, object detection, instance segmentation.

🛠️ Toolbox Features

Patch Annotation	Rectangle Annotation	(Multi) Polygon Annotation
Image Classification	Object Detection	Instance Segmentation
Segment Anything (SAM)	Polygon Classification	Region-based Detection
Cut	Combine	Simplify
See Anything (YOLOE)	LAI Classification

Video Inference	Explorer

Enhance your CoralNet experience with these tools:

• 📥 Download: Retrieve Source data (images and annotations) from CoralNet
• 🎬 Rasters: Import images, or extract frames directly from video files
• ✏️ Annotate: Create annotations freely
• 👁️ Visualize: See CoralNet and CPCe annotations superimposed on images
• 🔬 Sample: Sample patches using various methods (Uniform, Random, Stratified)
• 🧩 Patches: Create patches (points)
• 🔳 Rectangles: Create rectangles (bounding boxes)
• 🟣 Polygons: Create polygons (instance masks)
  • 👨‍👩‍👧‍👦 MultiPolygons: Combine multiple, non-overlapping polygons (i.e, genets)
• ✍️ Edit: Cut and Combine polygons and rectangles
• 🦾 SAM: Use FastSAM, CoralSCOP, RepViT-SAM, EdgeSAM, MobileSAM, and SAM to create polygons
  • Uses xSAM
• 👷‍♂️ Work areas: Perform region-specific detections / segmentations with any model
• 👀 YOLOE (See Anything): Detect similar appearing objects using visual prompts automatically
• 🧪 AutoDistill: Use AutoDistill to access the following for creating rectangles and polygons:
  • Uses Grounding DINO, OWLViT, OmDetTurbo
• 📻 Tune: Tune hyperparameters to identify ideal training conditions
• 🧠 Train: Build local patch-based classifiers, object detection, and instance segmentation models
• 🧙‍♂️ Deploy: Use trained models for predictions
• 📊 Evaluation: Evaluate model performance
• 🚀 Optimize: Productionize models for faster inferencing
• ⚙️ Batch Inference: Perform predictions on multiple images, automatically
• 🎞️ Video Inference: Perform predictions on a video in real-time, record the output and analytics
• 🔮 Explorer: Cluster, view, and re-label annotations using embeddings, mapped from feature-space
• ↔️ I/O: Import and Export annotations from / to CoralNet, Viscore, and TagLab
  • Export annotations as GeoJSONs, segmentation masks
• 📸 YOLO: Import and Export YOLO datasets for machine learning
• 🧱 Tile Dataset: Tile existing Detection / Segmentation datasets
  • Uses yolo-tiling
• 🏗️ Tile Inference: Pre-compute multiple work areas for an entire image

📝 TODO

• 🤗 Model Zoo: Download Ultralytics models from HuggingFace for use in toolbox
• 🦊 BioCLIP, MobileCLIP (AutoDistill): Automatically classify annotations
• 📦 Toolshed: Access tools from the old repository

💾 How to Install

🐍 Anaconda

It's recommended to use Anaconda to create an environment for the toolbox:

# cmd

# Create and activate an environment
conda create --name coralnet10 python=3.10 -y
conda activate coralnet10

📦 Install

Once this has finished, install the toolbox using uv:

# cmd

# Install uv first
pip install uv

# Install with uv
uv pip install coralnet-toolbox

Although fast, uv is still relatively new; if this fails, simply fall back to using pip:

# cmd

# Install
pip install coralnet-toolbox

⚡ CUDA

If you have CUDA, you should install the versions of cuda-nvcc and cudatoolkit that you need, and then install the corresponding versions of torch and torchvision. Below is an example of how that can be done using CUDA version 11.8:

# cmd

# Example for CUDA 11.8
conda install nvidia/label/cuda-11.8.0::cuda-nvcc -y
conda install nvidia/label/cuda-11.8.0::cuda-toolkit -y

# Example for torch w/ CUDA 11.8
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118 --upgrade

If CUDA is installed on your computer, and torch was built with it properly, you should see a 🐇 icon in the toolbox instead of a 🐢; if you have multiple CUDA devices available, you should see a 🚀 icon, and if you're using a Mac with Metal, you should see an 🍎 icon (click on the icon to see the device information).

See here for more details on versions for the following:

• cuda-nvcc
• cudatoolkit
• torch

▶️ Run

Finally, you can run the toolbox from the command line:

# cmd

# Run
coralnet-toolbox

How to Upgrade

When opening the toolbox, you will be notified if there is an update available, and you have the option to do so, if you so choose. To upgrade, run the following command from your terminal:

# cmd

uv pip install -U coralnet-toolbox==[enter_newest_version_here]

Again, fall back to using just pip and not uv if this fails.

🌊 Success Stories

Using the toolbox? Share your successes and we can feature your work to help others learn!

🐠 About CoralNet

Coral reefs are vital ecosystems that support a wide range of marine life and provide numerous benefits to humans. However, they are under threat due to climate change, pollution, overfishing, and other factors. CoralNet is a platform designed to aid researchers and scientists in studying these important ecosystems and their inhabitants.

CoralNet allows users to upload photos of coral reefs and annotate them with detailed information about the coral species and other features present in the images. The platform also provides tools for analyzing the annotated images, and create patch-based image classifiers.

The CoralNet-Toolbox is an unofficial tool developed to augment processes associated with analyses that use CoralNet and Coral Point Count (CPCe).

🏁 Conclusion

In summary, this repository provides a range of tools that can assist with interacting with CoralNet and performing various tasks related to analyzing annotated images. These tools can be useful for researchers and scientists working with coral reefs, as well as for students and hobbyists interested in learning more about these important ecosystems.

📝 Citation

If used in project or publication, please attribute your use of this repository with the following:

@misc{CoralNet-Toolbox,
  author = {Pierce, Jordan and Edwards, Clinton and Rojano, Sarah and Cook, Sophie and Sweeney, Edward and Costa, Bryan and Vieham, Shay and Battista, Tim},
  title = {CoralNet-Toolbox},
  year = {2023},
  howpublished = {\url{https://github.com/Jordan-Pierce/CoralNet-Toolbox}},
  note = {GitHub repository}
}

---

⚠️ Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an 'as is' basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

📝 License

Software code created by U.S. Government employees is not subject to copyright in the United States (17 U.S.C. §105). The United States/Department of Commerce reserve all rights to seek and obtain copyright protection in countries other than the United States for Software authored in its entirety by the Department of Commerce. To this end, the Department of Commerce hereby grants to Recipient a royalty-free, nonexclusive license to use, copy, and create derivative works of the Software outside of the United States.