Skip to main content

H2MARE - Geospatial Processing for Climate and Ocean Data

Python PyPI Docs

A Python pipeline for downloading and preprocessing multi-source oceanographic and atmospheric data into analysis-ready formats. H2MARE streamlines the acquisition and harmonization of data from major climate and ocean observation services, optimized for large-scale spatiotemporal analysis.

Features

  • Multi-source data integration: Download and process data from CMEMS, AVISO, and ERA5.
  • Format conversion: Automated conversion from NetCDF/GRIB to optimized Zarr and Parquet formats.
  • Data compilation: Regrid and interpolate multi-resolution datasets to a common grid.
  • Point and geometry extraction: Extract time series for specific locations or spatial features.

Data Sources

H2MARE supports the following data providers. API keys and authentication are required for each.

  • CMEMS — Copernicus Marine Service: satellite and in-situ ocean observations
  • AVISO — Archiving, Validation and Interpretation of Satellite Oceanographic data
  • CDS-ERA5 — ERA5 hourly atmospheric reanalysis (1940–present)

Refer to each provider's documentation for authentication setup before use.

Installation

Prerequisites

  • Python >= 3.11
  • uv — fast Python package and project manager

Install from PyPI

pip install h2mare
# or
uv add h2mare

That installs the pipeline: download, convert, compile, extract, Parquet. The plotting helpers (h2mare.utils.plot, ParquetIndexer.plot) need the viz extra, which adds cartopy, matplotlib and plotly:

pip install "h2mare[viz]"
# or
uv add "h2mare[viz]"

Install from source

git clone https://github.com/h2ugoparra/h2mare.git
cd h2mare
uv sync

Configuration

H2MARE requires two configuration files in your working directory before first use.

1. config.yaml

Defines variables, dataset IDs, bounding boxes, and processing parameters.

Start from config.example.yaml — copy it to config.yaml and edit. It holds four entries chosen to cover the shapes you are likely to need: a plain 2-D daily variable, a 3-D one with depth levels, an hourly packed ERA5 one, and the h2ds entry the compile step writes to.

The repository's own config.yaml configures 16 variables and is a reference, not a starting point — useful for seeing how a particular source is wired up, too heavy to copy wholesale. Every field is documented in the configuration reference.

2. .env

# Path to external or large-capacity storage for processed Zarr files
STORE_ROOT=/path/to/your/storage

# AVISO credentials (required for FSLE, Eddies)
AVISO_USERNAME=your_username
AVISO_PASSWORD=your_password
AVISO_FTP_SERVER=ftp-access.aviso.altimetry.fr

AVISO credentials cross the network in cleartext. The AVISO server offers plain FTP only — it refuses AUTH TLS ("500 AUTH not understood", checked 2026-09-01) — so there is no FTPS option to enable. Use a password unique to AVISO, and prefer running downloads from a trusted network.

CMEMS credentials are configured via the copernicusmarine client. ERA5 / CDS credentials are configured via the cdsapi client. See the CDS documentation for setup.

Note: Both files must be present in the directory where you run h2mare. You can also set the H2MARE_ROOT environment variable to point to a different directory containing them.

Quick Start

# Download and process a single variable for a specific date range
uv run h2mare run -v sst --start-date 2021-01-01 --end-date 2021-12-31

# Multiple variables at once
uv run h2mare run -v seapodym -v mld -v o2 -v chl

# Infer missing dates from the existing store and download what's new
uv run h2mare run -v sst

# Download only (skip Zarr conversion)
uv run h2mare run -v sst --no-convert

# Validate configuration without downloading
uv run h2mare run -v sst --dry-run

# Process all configured variables
uv run h2mare run

# Check the stores for days that went missing without failing anything
uv run h2mare audit --all

Coverage elsewhere in the pipeline is a frontier — a start date and an end date — so a year holding January 1st and December 31st reports itself complete however much is missing in between. audit reads the time axes and reports the holes. See the CLI docs for the value check and for known_gaps, which excludes days the provider never published.

Development

# Run the full test suite
uv run pytest tests/

# Run a single test file
uv run pytest tests/test_zarr_catalog.py -v

# Lint and format
uv run ruff check h2mare/
uv run ruff format h2mare/

Built with

Library Role
xarray N-dimensional labelled arrays and NetCDF/Zarr I/O
zarr Chunked, compressed array storage
dask Parallel and out-of-core computation
polars Fast DataFrame engine for Parquet I/O
duckdb In-process SQL for Parquet overlap resolution and scanning
geopandas Geometry-based spatial extraction
plotly Interactive time-series and spatial visualizations
copernicusmarine CMEMS dataset access
cdsapi ERA5 / CDS dataset access

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests on GitHub.

To report a security vulnerability, please do not open a public issue — see SECURITY.md.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

This project was developed under the framework of COSTA project. This project relies on data from Copernicus Marine Service, AVISO, Copernicus Climate Data Store, and NOAA NCEI. We gratefully acknowledge these organizations for providing open access to their datasets.

Metadata

Release files for h2mare 0.10.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for h2mare 0.10.3
File Size Uploaded Provenance Actions
h2mare-0.10.3.tar.gz 536.5 kB With provenance Details for h2mare-0.10.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for h2mare 0.10.3
File Interpreter ABI Platform Actions
h2mare-0.10.3-py3-none-any.whl Python 3 none any Details for h2mare-0.10.3-py3-none-any.whl

Total release size: 873.1 kB

Release files / h2mare-0.10.3.tar.gz

Download URL h2mare-0.10.3.tar.gz
Size 536.5 kB
Tags Source
SHA-256 checksum
How to use checksums
e1408e960de99f8f19110c853ae09313220d0d9b9cf29001855cc6f995d63752
BLAKE2b-256 checksum
How to use checksums
0e3fbee5625a1a6f6d7c3336ce0d283d15dce5341ac2f7fb6ece2d78790e68bb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

Release files / h2mare-0.10.3-py3-none-any.whl

Download URL h2mare-0.10.3-py3-none-any.whl
Size 336.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
160dd2f02f988adf396431f962926a632b723794dd3127e59350830529451ba9
BLAKE2b-256 checksum
How to use checksums
303851d2f79291668fab8eb0258d727c2874fb971f0e44a8d530e3534c0c9dfa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

Security for h2mare 0.10.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for h2mare-0.10.3.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 9, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page