bigearthnet-common 0.1.1

A collection of common tools to interact with the BigEarthNet dataset.

Common BigEarthNet Tools

A personal collection of common tools to interact with the BigEarthNet dataset.

This library provides a collection of high-level tools to better work with the BigEarthNet dataset.

ben_common tries to accomplish three goals:

1. Collect the most relevant constants into a single place to reduce the time spent looking for these, like:
   • The 19 or 43 class nomenclature strings
   • URL
   • Band statistics (mean/variance) as integer and float
   • Channel names
   • etc.
2. Provide parsing functions to convert the metadata json files to a geopandas GeoDataFrame's.
   • Allow for easy top-level statistical analysis of the data in a familiar pandas-style
   • Provide functions to enrich GeoDataFrames with often required BigEarthNet metadata (like the season or country of the patch)
3. Simplify the building procedure by providing a command-line interface with reproducible results

Deep Learning

One of the main purposes of the dataset is to allow deep learning researchers and practitioners to train their model on multi-spectral satellite data easily. In that regard, there is a general recommendation to drop patches that are covered by seasonal snow or clouds. Also, the novel 19-class nomenclature should be preferred over the original 43-class nomenclature. As a result of these recommendations, some patches have to be excluded from the original raw BigEarthNet dataset that is provided at BigEarthNet. This is especially important for higher-level statistical analysis.

To simplify the procedure of pre-converting the json metadata files, the library provides a single command that will generate a recommended GeoDataFrame with extra metadata (country/season data of each patch) while dropping all patches that are not recommended for deep learning research.

To generate such a GeoDataFrame and store it as an parquet file, use:

• gdf_builder build-recommended-parquet (available after installing package) or
• python -m bigearthnet_common.gdf_builder build-recommended-parquet

If you want to read the raw json files and convert those to a GeoDataFrame file, without dropping any patches or adding any metadata, use:

• gdf_builder build-raw-ben-parquet (available after installing package) or
• python -m bigearthnet_common.gdf_builder build-raw-ben-parquet

Local Installation

Use just to install the package or run steps from justfile directly. Requires mamba (highly recommended) or poetry to be installed.

Local Documentation

{% include note.html content='Building and serving the documentation requires Docker to be installed!' %} After creating the ben_common_env environment, run

docker-compose up

Or with just:

just docs

After running the command, the documentation server will be available at

• http://0.0.0.0:4000/bigearthnet_common/ or
• http://localhost:4000/bigearthnet_common/ (WSL).

To review the source code, please look at the corresponding ipynb notebook from the nbs folder. These notebooks include extensive documentation, visualizations, and tests. The automatically generated Python files are available in the bigearthnet_common module.