bigearthnet-common 0.3.0

A collection of common tools to interact with the BigEarthNet dataset.

Common BigEarthNet Tools

A personal collection of common tools to interact with the BigEarthNet dataset.

This library provides a collection of high-level tools to better work with the BigEarthNet dataset.

ben_common tries to accomplish three goals:

1. Collect the most relevant constants into a single place to reduce the time spent looking for these, like:
   • The 19 or 43 class nomenclature strings
   • URL
   • Band statistics (mean/variance) as integer and float
   • Channel names
   • etc.
2. Provide parsing functions to convert the metadata JSON files to a geopandas GeoDataFrame.
   • Allow for easy top-level statistical analysis of the data in a familiar pandas-style
   • Provide functions to enrich GeoDataFrames with often required BigEarthNet metadata (like the season or country of the patch)
3. Simplify the building procedure by providing a command-line interface with reproducible results

Installation

I strongly recommend to use mamba or conda with miniforge to install the package with:

• mamba/conda install bigearthnet-common -c conda-forge

As the bigearthnet_common tool is built on top of geopandas the same restrictions apply. For more details please review the geopandas installation documentation.

The package is also available via PyPI and could be installed with:

• pip install bigearthnet_common (not recommended)

TL;DR

The most relevant functions are exposed as CLI entry points. To quickly search for BigEarthNet constants of interest, call:

• ben_constants_prompt or
• python -m bigearthnet_common.constants

To build the tabular data, use:

• ben_gdf_builder --help or
• python -m bigearthnet_common.ben_gdf_builder --help

Deep Learning

One of the primary purposes of the dataset is to allow deep learning researchers and practitioners to train their models on multi-spectral satellite data. In that regard, there is a general recommendation to drop patches that are covered by seasonal snow or clouds. Also, the novel 19-class nomenclature should be preferred over the original 43-class nomenclature. As a result of these recommendations, some patches have to be excluded from the original raw BigEarthNet dataset that is provided at BigEarthNet.

To simplify the procedure of pre-converting the JSON metadata files, the library provides a single command that will generate a recommended GeoDataFrame with extra metadata (country/season data of each patch) while dropping all patches that are not recommended for deep learning research.

To generate such a GeoDataFrame and store it as an parquet file, use:

• ben_gdf_builder build-recommended-parquet (available after installing package) or
• python -m bigearthnet_common.gdf_builder build-recommended-parquet

If you want to read the raw JSON files and convert those to a GeoDataFrame file without dropping any patches or adding any metadata, use:

• ben_gdf_builder build-raw-ben-parquet (available after installing package) or
• python -m bigearthnet_common.gdf_builder build-raw-ben-parquet

Contributing

Contributions are always welcome!

Please look at the corresponding ipynb notebook from the nbs folder to review the source code. These notebooks include extensive documentation, visualizations, and tests. The automatically generated Python files are available in the bigearthnet_common module.

More information is available in the contributing guidelines document.