Skip to main content

era5_to_int

A simple utility for converting ERA5 netCDF files to the WPS intermediate format.

This has been packaged up and slightly modified from the original NCAR era5_to_int scripts. Huge thanks to NCAR and to Michael Duda and Anthony Islas for writing the scripts!

Installation

You can install the command line utility in a variety of ways. One of the easiest and most common is to use pipx:

pipx install era5_to_int

Overview

The era5_to_int utility converts ERA5 pressure-level netCDF files to the WRF Pre-processing System (WPS) intermediate file format, permitting the use of these data with either the Weather Research and Forecasting (WRF) model or the Model for Prediction Across Scales - Atmosphere (MPAS-A).

The required command-line arguments are the path to the local files and the start and end datetimes of ERA5 files to convert. For example:

era5_to_int /root/path/to/era5/ 2024-05-01_00 2024-05-02_00

or if you're using a different time format, then you'll need to put quotes around the datetimes:

era5_to_int /root/path/to/era5/ "2024-05-01 00" "2024-05-02 00"

The directory structure must be the same as NCAR's ERA5 AWS S3 bucket. This is the recommended place to download the ERA5 data since it's free and fast.

Conversion of a range of datetimes is possible through the use of additional command-line arguments as described in the Usage section.

The following surface fields from the d633 datasets are handled by the script:

Field Dataset Horiz. grid Num. levels
SOILGEO d633000 0.25-deg Lat-Lon 1
SP d633006 0.25-deg Lat-Lon 1
MSL d633000 0.25-deg Lat-Lon 1
VAR_2T d633000 0.25-deg Lat-Lon 1
VAR_2D d633000 0.25-deg Lat-Lon 1
VAR_10U d633000 0.25-deg Lat-Lon 1
VAR_10V d633000 0.25-deg Lat-Lon 1
RSN d633000 0.25-deg Lat-Lon 1
SD d633000 0.25-deg Lat-Lon 1
LSM d633000 0.25-deg Lat-Lon 1
SSTK d633000 0.25-deg Lat-Lon 1
SKT d633000 0.25-deg Lat-Lon 1
SWVL1 d633000 0.25-deg Lat-Lon 1
SWVL2 d633000 0.25-deg Lat-Lon 1
SWVL3 d633000 0.25-deg Lat-Lon 1
SWVL4 d633000 0.25-deg Lat-Lon 1
STL1 d633000 0.25-deg Lat-Lon 1
STL2 d633000 0.25-deg Lat-Lon 1
STL3 d633000 0.25-deg Lat-Lon 1
STL4 d633000 0.25-deg Lat-Lon 1
CI d633000 0.25-deg Lat-Lon 1

When pressure-level (isobaric) ERA5 files are being processed, the following atmospheric fields are handled:

Field Dataset Horiz. grid Num. levels
Z d633000 0.25-deg Lat-Lon 37
Q d633000 0.25-deg Lat-Lon 37
T d633000 0.25-deg Lat-Lon 37
U d633000 0.25-deg Lat-Lon 37
V d633000 0.25-deg Lat-Lon 37

As fields are converted from netCDF to intermediate format, their names are also converted to match WPS and MPAS-A expectations.

Downloading and spatial clipping the ERA5 data

This command line utility isn't very helpful if you don't have the ERA5 data. There is a docker image made specifically to download the ERA5 data from the AWS S3 bucket maintained by NCAR. Then the container will clip the global files to a user-defined region to substantially shrink the files. It will save the clipped files to a directory or optionally upload it to another S3 system defined by the user.

You can find the github repo here: https://github.com/envlib/era5-download

It will download all of the files necessary to run the era5_to_int utility.

Usage

The required command-line arguments are the path to the local files and the start and end datetimes of ERA5 files to convert. For example:

era5_to_int /root/path/to/era5/ 2024-05-01_00 2024-05-02_00

Upon successful completion, an intermediate file with the prefix ERA5 is created in the current working directory; for example, ERA5:2024-05-01_00.

By default, fields are converted at a six-hourly interval, and a different interval may be specified with the option -h. For example, to convert data at a three-hourly interval for the month of May 2024:

era5_to_int -h 3 /root/path/to/era5/ 2024-05-01_00 2024-05-02_00

Also by default, all variables required to run WRF are processed. If only a selection of variables is required, then use the --variables option:

era5_to_int --variables LANDSEA,SEAICE,SKINTEMP /root/path/to/era5/ 2024-05-01_00 2024-05-02_00

If you want all variables except a few, use --skip-vars (composes with --variables if both are given). Skipped variables do not need their ERA5 NetCDF files on disk, so this is useful when sourcing a field from elsewhere — for example, providing SST and sea ice from a higher-resolution product:

era5_to_int --skip-vars SST,SEAICE /root/path/to/era5/ 2024-05-01_00 2024-05-02_00

Usage help is provided by running the era5_to_int with the --help argument.

Incomplete input

Since 0.5.0, every input is resolved before anything is written. For each time in the window and each variable to be converted, era5_to_int finds the ERA5 file and the record holding that time. If any file is missing, any file lacks a needed time, or any file cannot be read, it lists every problem on stderr (each file once, with the times it affects) and exits 1 without writing an intermediate file.

Before 0.5.0, a missing file stopped the run mid-frame with the message on stdout, leaving a short last intermediate file, and a time absent from an existing file was silently written from the file's first record, that is, another time's data. The invariant files (land-sea mask, surface geopotential) hold a single 1979 record, which is used for every time by design.

Metadata

Release files for era5_to_int 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for era5_to_int 0.5.0
File Size Uploaded Provenance Actions
era5_to_int-0.5.0.tar.gz 26.8 kB Without provenance Details for era5_to_int-0.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for era5_to_int 0.5.0
File Interpreter ABI Platform Actions
era5_to_int-0.5.0-py3-none-any.whl Python 3 none any Details for era5_to_int-0.5.0-py3-none-any.whl

Total release size: 53.1 kB

Release files / era5_to_int-0.5.0.tar.gz

Download URL era5_to_int-0.5.0.tar.gz
Size 26.8 kB
Tags Source
SHA-256 checksum
How to use checksums
fd3fa08dc55f7d29820854ea25f8fa802c510f7b3bf2993a300c56bc8361943c
BLAKE2b-256 checksum
How to use checksums
f30a85a6d2b70c0c18c51d26d9138aa9a84dd4e7282b9d815f59d6fe9874eecb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.7

Release files / era5_to_int-0.5.0-py3-none-any.whl

Download URL era5_to_int-0.5.0-py3-none-any.whl
Size 26.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3af864f34a056aa9442caa681ae4de9a1e070d2fec018e28f567bb64559481cc
BLAKE2b-256 checksum
How to use checksums
bbd4d2401f5aac027ece55bf34836e32a97f4b5177191883dc72250ec38290e0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.7

Security for era5_to_int 0.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page