geobench 1.0.0

A benchmark designed to advance foundation models for Earth monitoring, tailored for remote sensing. It encompasses six classification and six segmentation tasks, curated for precision and model evaluation. The package also features a comprehensive evaluation methodology and showcases results from 20 established baseline models.

GEO-Bench: Toward Foundation Models for Earth Monitoring

GEO-Bench is a ServiceNow Research project.

GEO-Bench is a General Earth Observation benchmark for evaluating the performances of large pre-trained models on geospatial data. Read the full paper for usage details and evaluation of existing pre-trained vision models.

Installation

You can install GEO-Bench with pip:

pip install geobench

Note: Python 3.9+ is required.

Downloading the data

Set $GEO_BENCH_DIR to your preferred location. If not set, it will be stored in $HOME/dataset/geobench.

Next, use the download script. This will automatically download from Hugging Face

Run the command:

geobench-download

You need ~65 GB of free disk space for download and unzip (once all .zip are deleted it takes 57GB). If some files are already downloaded, it will verify the md5 checksum. Feel free to restart the downloader if it is interrupted.

Test installation

You can run tests. Note: Make sure the benchmark is downloaded before launching tests.

pip install pytest

geobench-test

Loading Datasets

See example_load_dataset.py for how to iterate over datasets.

import geobench

for task in geobench.task_iterator(benchmark_name="classification_v1.0"):
    dataset = task.get_dataset(split="train")
    sample = dataset[0]
    for band in sample.bands:
        print(f"{band.band_info.name}: {band.data.shape}")

Visualizing Results

See the notebook baseline_results.ipynb for an example of how to visualize the results.