miranda 0.3.0

Python utilities for climate data collection and management

Miranda

Python utilities for climate data collection, conversion, and management

• Documentation:

• Free Software:

Features

Data collection functions for climate and forecast data hosted at:

• ECMWF (ERA5, ERA5-Land, TIGGE)

• ECCC (Canada) (Monthly Climate Summaries, ECCC GEOAPI - In development)

• NCAR (CORDEX-NA on AWS)

Data conversion for Climate and Forecasting (CF) Variable and Metadata compliance:

• ECMWF (ERA5, ERA5-Land, TIGGE - In Development)

• ECCC (Canada) (Flat File Observations, Monthly Climate Summaries, Adjusted and Homogenized Climate Data, ECCC GEOAPI - In Development)

• MELCC (Québec)

• Hydro-Québec (In Development)

Database structuring and facets validation:

• Simulations:

  • WCRP (CMIP5, CMIP6, CMIP5-CORDEX, CORDEX-ADJUST, ISIMIP, etc.)

• Station-Observations:

  • MELCC (Québec) (Needs mdbtools installed)

  • ECCC (Canada) (In Development)

  • Hydro-Québec (In Development)

• Gridded-Observations:

  • NRCAN (Canada) (Future)

  • MELCC (Future)

• Reanalyses:

  • ECMWF (ERA5, ERA5-Land, TIGGE)

  • NASA (DayMET, AgMerra/AgCFSR, MERRA2) - In Development

  • NCEP (CFSR/CFSv2) - In Development

  • WFDEI-GEM-CaPa (University of Saskatchewan) - In Development

Installation

miranda is not yet available on PyPI, so the suggested method to install is as follows:

$ git clone [e-mail]:Ouranosinc/miranda.git
$ cd miranda

# If using Anaconda:
$ conda create -n miranda -f environment.yml
$ conda activate miranda

$ pip install miranda[full]

miranda also relies on PyESSV for its climate data controlled vocabulary. This library requires additional installation steps:

$ mkdir -p ~/.esdoc
$ git clone [e-mail]:ES-DOC/pyessv-archive.git ~/.esdoc/pyessv-archive

We strongly suggest using Anaconda3/miniconda3 (with the conda-forge repository enabled) to manage your environment and dependencies

• Anaconda: https://www.anaconda.com/products/distribution

• Miniconda: https://docs.conda.io/en/latest/miniconda.html

• conda-forge: https://conda-forge.org/#about

Contributing

See the contributing documentation: https://miranda.readthedocs.io/en/latest/contributing.html

Documentation

The full documentation is at https://miranda.readthedocs.io/en/latest.

History

0.3.0 (2022-11-24)

Contributors to this version: Trevor James Smith (@Zeitsperre), Pascal Bourgault (@aulemahal), David Huard (@huard), Travis Logan (@tlogan2000), Gabriel Rondeau-Genesse (@RondeauG), and Sébastien Biner (@sbiner).

Announcements

• First public release on PyPI.

New features

• Dataset conversion tools (miranda.convert) use a JSON-definition file to dynamically populate metadata, run data quality checks, and convert units to CF-compliant standard. Supported datasets are:

  • ERA5/ERA5-Land (complete)

  • MELCC (stations) (beta)

  • ECCC (stations) (alpha)

  • NASA DayMet (WIP)

  • NASA AgMerra/AgCFSR (WIP)

  • Hydro Québec (stations) (WIP)

  • DEH (stations) (WIP)

  • WFDEI-GEM-CAPA (WIP)

• Module (miranda.eccc) for ECCC station data and ECCC Adjusted and Homogenized Canadian Climate Data (AHCCD) conversion (WIP).

• Module (miranda.ncar) for fetching interpolated CORDEX-NAM (22i/44i) from NCAR AWS data storage.

• Module (miranda.ecmwf) for fetching ECMWF ERA5/-Land (single-levels, pressure-levels, monthly-means) datasets via CDSAPI.

• Module (miranda.gis) for setting specific subsetting domains used when converting gridded datasets.

• Modules (miranda.archive and miranda.remote) for performing data archiving actions locally and remotely (powered by fabric and paramiko) (WIP).

• Module (miranda.decode) for ingesting and parsing dataset metadata based on filename and dataset attributes. Supported datasets are:

  • miranda converted datasets

  • CMIP6

  • CMIP5

  • CMIP5-CORDEX

  • ISIMIP-FT

  • CanDCS-U6 (PCIC)

• Module (miranda.structure) for create constructing file-tree databases based on YAML-defined metadata schemas (WIP).

• Modules (miranda.cv and miranda.validators) for validating metadata using ESGF controlled vocabularies (taken from pyessv-archive) and schema definitions (powered by schema), respectively (WIP).