langrs 1.1.0

Remote sensing image segmentation using LangSAM

LangRS

A python package that omptimizes zero-shot segmentation of aerial image based with GroundingDINO and Segment Anything Model (SAM)

Introduction

LangRS is a Python package for remote sensing image segmentation, it is built on top of the Segment-Geospatial package. It combines advanced techniques like bounding box detection, semantic segmentation, and outlier rejection to deliver precise and reliable segmentation of geospatial images.

Citation

@article{DIAB2025100105,
title = {Optimizing zero-shot text-based segmentation of remote sensing imagery using SAM and Grounding DINO},
journal = {Artificial Intelligence in Geosciences},
volume = {6},
number = {1},
pages = {100105},
year = {2025},
issn = {2666-5441},
doi = {https://doi.org/10.1016/j.aiig.2025.100105},
url = {https://www.sciencedirect.com/science/article/pii/S2666544125000012},
author = {Mohanad Diab and Polychronis Kolokoussis and Maria Antonia Brovelli},
keywords = {Foundation models, Multi-modal models, Vision language models, Semantic segmentation, Segment anything model, Earth observation, Remote sensing},
}

How it works

📊 Package Performance vs Ground Truth

🔄 Direct Comparison with SAMGEO Package

Features

• Bounding Box Detection: Locate objects in remote sensing images with a sliding window approach.
• Outlier Detection: Apply various statistical and machine learning methods to filter out anomalies in the detected objects based on the area of the detected bounding boxes.
• Non-Max Suppression Applies NMS to the input bounding boxes, can reduce accuracy slightly, but greatly increases inference speed and lowers memory usage.
• Area Calculation: Compute and rank bounding boxes by their areas.
• Image Segmentation: Detect and extract objects based on text prompts using LangSAM.

---

Installation

Install LangRS with pip

pip install langrs

Usage

Here is an example of how to use the LangRS class for remote sensing image segmentation:

from langrs import LangRS

# The class accepts tif/ RGB images
text_input = "object.tif" 

# Path to the input remote sensing image
image_input = "path_to_your_tif_file"

# Initialize LangRS with the input image, text prompt, and output directory
langrs = LangRS(image_input, text_input, "output_folder")

# Detect bounding boxes using the sliding window approach with example parameters
bounding_boxes = langrs.generate_boxes(window_size=600, overlap=300, box_threshold=0.25, text_threshold=0.25)

# Apply outlier rejection to filter anomalous bounding boxes
# This will return a dict with the follwing keys:
# ['zscore', 'iqr', 'svm', 'svm_sgd', 'robust_covariance', 'lof', 'isolation_forest']
# The value of each key represent the boudning boxes from the previous step with the 
# outlier rejection method of the key's name applied to them
bboxes_filtered = langrs.outlier_rejection()

# Retreive certain bounding boxes 
bboxes_zscore = bboxes_filtered['zscore']

# Generate segmentation masks for the filtered bounding boxes of the provided key
masks = langrs.generate_masks(boxes=bounding_boxes)
# Or
masks = langrs.generate_masks(boxes=bboxes_zscore)

Input Parameters for LangRS Methods

LangRS Initialization:

• image: Path to the input image.
• prompt: Text prompt for object detection.
• output_path: Directory to save output files.

generate_boxes:

• window_size (int): Size of each chunk for processing. Default is 500.
• overlap (int): Overlap size between chunks. Default is 200.
• box_threshold (float): Confidence threshold for box detection. Default is 0.5.
• text_threshold (float): Confidence threshold for text detection. Default is 0.5.

outlier_rejection:

Applies multiple outlier detection methods (e.g., Z-Score, IQR, SVM, LOF) to filter bounding boxes.

generate_masks:

• boxes (list[torch.tensor]): The input boxes, the model will segment what is inside these boxes only.

---

Output

When the code runs, it generates the following outputs:

1. Original Image with Bounding Boxes: Shows the detected bounding boxes.
2. Filtered Bounding Boxes: Bounding boxes after applying outlier rejection.
3. Segmentation Masks: Overlays segmentation masks on the original image.
4. Area Plot: A scatter plot of bounding box areas to visualize distributions.

The results are saved in the specified output directory, organized with a timestamp to separate runs.

---

Contributing

We welcome contributions! If you'd like to add features or fix bugs:

1. Fork the repository.
2. Create a feature branch.
3. Submit a pull request.

---

License

This project is licensed under the MIT License. See the LICENSE file for details.

---

Support

For any questions or issues, please open an issue on GitHub or contact the project maintainers.