geowatch 0.18.2

No project description provided

GeoWATCH is an open source research and production environment for image and video segmentation and detection with geospatial awareness.

This repository addresses the algorithmic challenges of the IARPA SMART (Space-based Machine Automated Recognition Technique) program. The goal of this software is analyze space-based imagery to perform broad-area search for natural and anthropogenic events and characterize their extent and progression in time and space.

The following table provides links to relevant resources for the SMART WATCH project:

The GeoWATCH Gitlab Repo	https://gitlab.kitware.com/computer-vision/geowatch/
Pypi	https://pypi.org/project/geowatch/
Read the docs	https://geowatch.readthedocs.io
Slides	Software Overview Slides and KHQ Demo Slides
The Phase 2 Internal SMART GeoWATCH DVC Data Repo	https://gitlab.kitware.com/smart/smart_data_dvc/
The Phase 2 Internal SMART GeoWATCH DVC Experiment Repo	https://gitlab.kitware.com/smart/smart_expt_dvc/

Purpose & Features

GeoWATCH can be used to train, predict, and evaluate segmentation models on multi-sensor image or video data. Polygons can be extracted or “tracked” across frames in a video to produce vectorized predictions.

Images can be in different resolutions, may be paired with geospatial metadata (but this is not required), and have any number of sensed or derived raster bands. Each raster band for an image need not be at the same resolution. The only requirement is that there is an affine transform that relates each underlying “asset space” into “image space”, which in turn must be similarly related to each “video space”. These transforms and all other details of the dataset are provided in a kwcoco file.

Dataloaders are setup to work with kwcoco files, and at train time details like mean/std computation, classes, frequency weights are handled automatically as opposed to common practice of hardcoding those values somewhere in a config file or in the code. In this way GeoWATCH seeks to run on the input data the user provides, rather than make assumptions about it. The only restriction is that the data must be registered in a kwcoco file, which is easy to do, can co-exist with arbitrary other on-disk structures, and has many benefits. Images can be arbitrarily large or small, and can be used in-situ (i.e. the raw images need not be pre-processed in any way), although some formats (e.g. COGs) will be more efficient than others.

Use Case: Heavy Construction

The motivating use-case for this software is detection of heavy construction events and the classification of their phases.

The first image illustrates the broad area search (BAS) component of the pipeline that uses low spatio-temporal resolution data to detect candidate “salient” regions for further processing.

The next main component of the system is activity characterization (AC) where higher resolution data is used to refine predicted information. In this case we classify each polygon as a different phase of construction. In the above example there are 3 detected candidates. We now zoom in on the one in the middle.

This shows the system detecting the construction of the KHQ building and classifying its phases. This demo was run on public data, and can be reproduced with Tutorial 6. The system was not trained on this region.

Publicly released model weights indexed via the decentralized IPFS protocol as well as through Kitware’s centralized Girder system. Note IPFS links use the ipfs.io gateway, but the CID can be used to accesss the data directly. See IPFS tutorials for details.

Publicly Released Model Weights
Release Date	IPFS	Girder
2024-01-11	bafybeiclo3c4bnhuumj77nxzodth442ybovw77cvbzp7ue23lsfnw4tyxa	65a94833d5d9e43895a66505

System Requirements

Before you start you must have installed Python. We currently support CPython versions 3.10 and 3.11.

Getting Started

The geowatch package is available on pypi and can be installed with pip. To install a barebones version of geowatch with limited features and dependencies, run:

pip install geowatch[headless]

Note that it is import to specify “headless”, to indicate that the opencv-python-headless variant of opencv that should be used. Alternatively, you could specify “graphics” to use the opencv-python variant, but we have found that this can cause conflicts with Qt libraries.

Alternatively, for a fully featured install of GeoWATCH run:

pip install geowatch[headless,optional,development,tests]

After installing geowatch from from pypi, you will have access to the GeoWATCH command line interface (CLI). At this point you should be able to use the CLI to list available commands:

geowatch --help

Unfortunately, the install is not complete. This is because binary wheels for GDAL are not available on pypi, and this means we cannot access them at GeoWATCH install-time. Fortunately, Kitware hosts binary GDAL wheels, and GeoWATCH provides a tool to install them and complete its installation.

geowatch finish_install

If you use the fully featured install command (which can be run even if GeoWATCH is already installed), or have at least installed xdoctest, you can test that your install is functioning correctly by running the doctests in the geowatch module:

xdoctest -m geowatch

The GeoWATCH CLI has support for tab-complete, but this feature needs to be enabled.

For more details see the installing GeoWATCH for development guide.

We also have limited windows support, see installing GeoWATCH on Windows.

Tutorials

We have a set of tutorials related to training models and predicting with the system.

• Tutorial 1: Toy RGB Fusion Model Example

• Tutorial 2: Toy MSI Fusion Model Example

• Tutorial 3: Feature Fusion Tutorial

• Tutorial 4: Misc Training Tutorial

• Tutorial 5: KR2 BAS SMART Demo

• Tutorial 6: KHQ SMART Demo

Documentation

For quick reference, a list of current documentation files is:

• Onboarding Docs

• Internal Resources

• The GeoWATCH CLI

• Contribution:

  • Contribution Instructions

  • Rebasing Procedure

  • Testing Practices

  • Supporting Projects

  • Coding Conventions

• Installing:

  • Installing GeoWATCH

  • Installing GeoWATCH on Windows

  • Installing Python via Conda

  • Installing Python via PyEnv

• Fusion Related Docs:

  • TA2 Fusion Overview

  • TA2 Deep Dive Info

  • TA2 Feature Integration

• Older Design Docs:

  • Structure Proposal

Development

For new collaborators, please refer to the onboarding docs

For internal collaborators, please refer to the internal docs

For more details about the GeoWATCH CLI and other CLI tools included in this package see: the GeoWATCH CLI docs

The geowatch module is built on top of several other supporting libraries developed by Kitware. Familiarity with these packages will make it easier to understand the GeoWATCH codebase. Particularly, developers should be have some familiarity with kwcoco, kwimage, scriptconfig, and ubelt. Also helpful is familiarity with ndsampler, delayed_image, cmd_queue, and xdoctest.

Related Work

There are other GIS and segmentation focused torch packages out there:

• TorchGeo - Torch geo provides many custom dataloaders for standard datasets. In contrast, we provide a single data loader for kwcoco files.

• Raster Vision - A similar framework. One major difference is that ours provides the ability to construct multi-sensor batches with heterogeneous resolutions (using the help of delayed_image).

• MMSegmentation - Contains standardized models with flexible model configuration. We use some the mmlabs models, but their library doesn’t have the data flexibility (e.g. large image support) that kwcoco provides.

Published Research

• IGARSS 2024: Slides, Conference Page, Paper (ArXiV), Paper (DeSci)

• WACV 2024: Open Access Paper.

Acknowledgement

This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), 6 Intelligence Advanced Research Projects Activity (IARPA), via 2021-2011000005. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein