cosmodules 1.1.0

No project description provided

COSMOduleS: Classification, Object detection, Segmentation MOduleS

Github: source

Introduction

This repo provides comprehensive preprocessing and post-processing tools for common Computer Vision tasks.

Tasks	Subtasks	Defined Format	Visualization	Format Conversion	Output Analysis	Label Merging	Active Learning
Classification	binary1 binary-bg2 multi-class1 multiclass-bg2 multi-binary3	single_label1 single_label_bg2 multi_label3	-	-	metrics plotting export	ALL	Entropy
Detection	-	coco voc yolo GENERAL	ALL	between ANY two types	metrics plotting export	V	horizontal flip
Segmentation	instance1 semantic2	coco1+2 GENERAL1+2	ALL	coco2general	metrics plotting export	-	instance semantic

• "bg" means background. If there is background class, it must be class 0 in this repo.
• Adding prediction results after the defined format can use the visualization and output analysis. All the formats with predictions are in example/*/prediction, e.g. here.

Motivation

• [Classification] Complicated tasks

task	label idx min	compute class-0 metrics	threshold optimization	data format
binary classification	0	V	V	single_label
binary classification (cls-0 background)	1		V	single_label_background
multi-class classification	0	V		single_label
multi-class classification (cls-0 background)	1		V	single_label_background
multi-label classification (cls-0 background)	0		V	multi_label

• [Classification] threshold optimization

  • multi-class classification (cls-0 background) checks whether prob-cls-0 < threshold, if yes, the pd-cls is pd[1:].argmax()
  • multi-class classification (cls-0 background) and multi-label classification (cls-0 background) take the mean of all optimized threshold for each foreground class

• [Object Detection] Develop a GENERAL format to be the most convenient.

The formats can be summarized as following:

format	extension	files	type	box	disadvantage
coco	.json	1	int	(xmin, ymin, w, h)	get label of an image
yolo	.txt	len(imgs)	float	(cx, cy, w/2, h/2)	visualization, compute metrics, etc.
voc	.xml	len(imgs)	int	(xmin, ymin, xmax, ymax)	get class list
general	.json	1	int	(xmin, ymin, xmax, ymax)	NO

• [Segmentation] Develop a GENERAL format to be the most convenient.

Includes	Content	Advantage
general.json	Includes every imgs: path, contour, filled and boxes with class	Searching
gt_contour_*.npy	(H, W) with {0, 1, ..., num_classes} int	Plotting
gt_filled_*.npy	(num_classes, H, W) with 0 or 1 int values	Compute IOU for Metrics
*.jpg	Raw data	-

• Segmentation prediction format: (num_classes, H, W) with 0~1 float values (probability). e.g. here

Installation

pip install cosmodules

or

git clone https://github.com/bnbsking/COSMOduleS.git
pip install -e .

Quick Start - Classification

• Output Analysis:
  • Please conform your data format as either of one
    • single_label
    • multilabel
    • single_label_background
  • The analysis pipeline is at here
  • See more in the example

from cosmodules.classification import ClassificationAnalysis

ClassificationAnalysis(
    ant_path = "example/classification/data/single_label.json",
    save_folder = "example/classification/output/single_label",
)

• Label Merging:

from cosmodules.classification import ClassificationLabelMerging

ClassificationLabelMerging(
    cfg_path_list = [
        "example/classification/data/single_label.json",
        "example/classification/data_another_labeler/single_label.json",
    ],
    save_path = f"example/classification/output/label_merging/single_label.json"
)

• Active Learning (see more in the example):

from cosmodules.classification import ClassificationActiveLearning

ClassificationActiveLearning(
    pred_path = "example/classification/prediction/single_label.json",
    save_path = "example/classification/output/active_learning/single_label.json",
    loss_name = "entropy"
)

Quick Start - Object detection

• Format Conversion (see more in the example)

from cosmodules.detection import coco2any

coco2any(
    tgt_foramt = "voc",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_folder = "example/detection/output/visualization_gt_conversion/coco2voc"
)

or

from cosmodules.detection import coco2general

coco2general(
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_path = "example/detection/output/visualization_gt_conversion/coco2general/general.json"
)

• Visualization (see more in the example)

from cosmodules.detection import show_coco

show_coco(
    img_name = "pic0.jpg",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json"
)

or

from cosmodules.detection import show_general

show_general(
    img_name = "pic0.jpg",
    ant_path = "example/detection/data/general.json",
)  # when the anntotation includes predictions it will be shown!

• Output Analysis
  • Please use the above Format conversion to change data format as general
  • The analysis pipeline is at here

from cosmodules.detection import DetectionAnalysis

DetectionAnalysis(
    ant_path = "example/detection/data/general.json",
    save_folder = "example/detection/output/metrics"
)

• Label Merging:

from cosmodules.detection import DetectionLabelMerging

DetectionLabelMerging(
    cfg_path_list = [
        "example/detection/data/general.json",
        "example/detection/data_another_labeler/general.json",
    ],
    save_path = "example/detection/output/label_merging/general.json",
    ties_handling = "union"
)

• Active Learning:

from cosmodules.detection import DetectionActiveLearningByHFlip

DetectionActiveLearningByHFlip(
    pred_path_1 = f"{ROOT}/example/detection/prediction/general.json",
    pred_path_2 = f"{ROOT}/example/detection/prediction/general_horizontal_flip.json",
    save_path = f"{ROOT}/example/detection/output/active_learning/general.json"
)

Quick Start - Segmentation

• Format Conversion (see more in the example)

from cosmodules.segmentation import coco2general

coco2general(
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json",
    save_folder = f"example/segmentation/output/visualization_gt_conversion/coco2general"
)

• Visualization (see more in the example)

from cosmodules.segmentation import show_coco

show_coco(
    img_name = "img1.jpg",
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json"
)   # when the anntotation includes predictions it will be shown!

or

from cosmodules.segmentation import show_general

show_general(
    img_name = "img1.jpg",
    ant_path = "example/segmentation/data/general/general.json"
)

• Output Analysis
  • Please use the above Format conversion to change data format as general
  • The analysis pipeline is at here

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/instance/general.json",
    save_folder = "example/segmentation/output/metrics/instance",
    task = "instance",
)

or

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/semantic/general.json",
    save_folder = "example/segmentation/output/metrics/semantic",
    task = "semantic"
)

• Active Learning:

from cosmodules.segmentation import (
    InstanceSegmentationActiveLearningByHFlip,
    SemanticSegmentationActiveLearning
)

InstanceSegmentationActiveLearningByHFlip(
    pred_path_1 = "example/segmentation/prediction/instance/general.json",
    pred_path_2 = "example/segmentation/prediction/instance_horizontal_flip/general.json",
    save_path = "example/segmentation/output/active_learning/instance.json"
)

or

SemanticSegmentationActiveLearning(
    pred_path = "example/segmentation/prediction/semantic/general.json",
    save_path = "example/segmentation/output/active_learning/semantic.json",
    loss_name = "entropy"
)

Examples

• [detection]: format conversion workflow

• detection visualization

• confusion

• prf curves

More

• Feel free to ask if you have any question.
• Notice not supported
  • segmentation general2coco
  • segmentation label merging

Updates

• v1.1.0:
  • Add docker environment
  • Improve classification and detection metrics pipeline
    • template design pattern
    • hybrid approach of
      • stateless (get / staticmethod / abstract method)
      • stateful (_set / instance method)
    • cleaner architecture of
      • pr curve
      • confusion matrices
      • threshold optimization
    • add unit tests
      • see in here

from cosmodules.utils.flow.metrics.classification_metrics import ClassificationMetricsFlow

class TestClassificationMetricsFlow:
    def test_run_single_label(self):
        obj = ClassificationMetricsFlow(
            num_classes = 2,
            labels = np.array([0, 0, 1]),
            predictions = np.array([
                [0.95, 0.05],
                [0.1, 0.9],
                [0.2, 0.8]
            ]),
            save_path = "/app/example/classification/output/metrics_new/single_label/metrics.json",
        )
        results = obj.run()
        print("Results:", results)

if __name__ == "__main__":
    test = TestClassificationMetricsFlow()
    test.test_run_single_label()

or

import numpy as np

from cosmodules.utils.flow.metrics.detection_metrics import DetectionMetricsFlow


class TestDetectionMetricsFlow:
    def test_run(self):
        obj = DetectionMetricsFlow(
            num_classes = 3,
            labels = [
                np.array([[1, 0, 23, 220, 228]]),
                np.array([[1, 8, 15, 715, 630], [2, 733, 64, 1174, 588]]),
                np.array([[1, 35, 7, 112, 114], [1, 171, 58, 263, 121], [2, 114, 39, 170, 118], [2, 259, 62, 326, 118]]),
            ],
            predictions = [
                np.array([
                    [0, 23, 220, 228, 0.95, 1],
                    [30, 90, 80, 150, 0.8, 2],
                ]),
                np.array([
                    [8, 15, 715, 630, 0.85, 1],
                    [733, 64, 1174, 588, 0.6, 1],
                ]),
                np.array([
                    [35, 7, 112, 114, 0.75, 1],
                    [171, 58, 211, 88, 0.45, 1],
                    [114, 39, 170, 118, 0.99, 2],
                ]),
            ],
            save_path = "/app/example/detection/output/metrics_new/metrics.json",
        )
        results = obj.run()
        print("Results:", results)


if __name__ == "__main__":
    test = TestDetectionMetricsFlow()
    test.test_run()

Acknowledgement

• Confusion Matrix reference here