PCoRaS

Python Package for Cloud Optimzed Raster and Spatio-temporal Data, following the STAC guidelines this repository is based Cloud optimized type of files and Raster2stac functions to create an STAC collections compatible with OpenEO backend, getting an output up to a 5D hypercube dataset for different spatio-temporal data, following a collection of local OpenEO processes to test the compability of the item.

Setup

pip install pcoras

Note: Conda support is in development

Getting started

Command-line interface

Convert a NetCDF file into a local Zarr data cube:

    python -m pcoras convert \
    --path ./data/precipitation.nc \
    --output-folder ./output \
    --output-name precipitation \
    --xdim x \
    --ydim y \
    --tdim time

Replace the input path and dimension names with those in your dataset. For example, a dataset with longitude and latitude dimensions may require --xdim lon --ydim lat.

These six arguments are required by the command-line parser. NetCDF (.nc) is the default input extension. S3 upload and STAC publication are disabled by default; enable them with --upload-s3-data and --publish-stac, respectively, and supply the corresponding storage or collection settings.

The --path option accepts one command-line value: a file or directory path. The co command is an alias for convert.

Display the available commands and conversion options:

python -m pcoras --help
python -m pcoras convert --help

Selected conversion options:

Option	Purpose
--variables pr tas	Select variables by name. Replace pr and tas with names from your dataset.
--file-ext .nc	Set the input file extension; defaults to .nc.
--crs 32632	Supply a CRS as an integer EPSG code appropriate for your dataset.
--chunking '{"time": 31}'	Supply chunk sizes as a JSON object, using your dataset's dimension names.

Options defined as flags, such as --upload-s3-data, are enabled by their presence; do not append True or False.

Python API

Check that the package can be imported and inspect its Python API:

from pcoras import pcoras

help(pcoras)

Conversion examples are available in the repository's test folder. They illustrate how to configure input datasets, output data cubes, metadata, and storage options.

Metadata

PCoRaS preserves attributes shared by the source datasets. You can supply additional attributes through extra_attr, for example:

extra_attr = {
    "keywords": ["climate", "precipitation", "Zarr"],
    "License": "MIT",
    "Producer": "Your organisation",
}

These are example dataset attributes. Set them to match the source data and its actual licensing and provenance.

STAC metadata describes the collection or item, including its title, description, providers, and licensing. The constructor accepts additional dataset attributes through extra_attr; The user must ensure that attibutes contains title, description, providers, License, Producer or links parameters. Inspect the generated STAC metadata to check that it describes your dataset correctly.

The dataset license may differ from the software license.

S3-compatible storage

For workflows that upload outputs to S3-compatible storage, configure the upload options and credentials. The following snippet defines example configuration values; it does not start an upload:

upload_s3_data = True
bucket_name = "pcoras-data"
bucket_file_prefix = "precipitation/v01"
aws_region = None
aws_access_key = ["AWS_ACCESS_KEY_ID"]
aws_secret_key = ["AWS_SECRET_ACCESS_KEY"]
s3_endpoint_url = "https://your-s3-endpoint.example.org"

Set the environment variables before running the workflow, and replace the bucket, prefix, endpoint, and region with values for your storage service.

For the CLI, enable --upload-s3-data and pass the storage settings through --bucket-name, --bucket-file-prefix, --aws-region, --aws-access-key, --aws-secret-key, and --s3-endpoint-url, as required by your storage service. The Python snippet above reads environment variables explicitly; it does not imply that the CLI reads them automatically.

Set stac_version to the STAC version supported by your workflow and target catalog.

Mindmap and Flowchart

Below, is a the current mindmap and flowchart follow by this repo

Mindmap

Flowchart

Not supported

Currently authentification to upload the STAC collection is not supported yet. However the code for request and publication is provided.

License

MIT.

Credits

This project was done thanks to the collaboration of the Institute of Earth Observation and the Center for Climate change and Transformation at Eurac Research

Thanks To Irma Ruiz Velasco for the great work on graphics and pcoras logo!