castle-ai 0.0.1

Distinguish behavioral clusters Toolbox

CASTLE

CASTLE integrates the strengths of visual foundation models trained on large datasets possess open-world visual concepts, including Segment Anything (SA), DeAOT, and DINOv2, for one-shot image segmentation and unsupervised visual feature extraction. Furthermore, CASTLE employs unsupervised multi-stage and/or multi-layer UMAP clustering algorithms to distinguish behavioral clusters with unique temporal variability.

Install

git clone https://github.com/RaisoLiu/castle-animal.git
cd castle-animal
python install .

Example

Image segmentation

1+1

Video objects segmentation

from castle import generate_aot

tracker = generate_aot(ckpt_path, MODEL, DEVICE)
tracker.add_reference_frame(frame, mask, num_object)


new_mask = 

Visual latent extractioin

1+1

UMAP & HDBSCAN analysis

1+1