era5_to_int
A simple utility for converting ERA5 netCDF files to the WPS intermediate format.
This has been packaged up and slightly modified from the original NCAR era5_to_int scripts. Huge thanks to NCAR and to Michael Duda and Anthony Islas for writing the scripts!
Installation
You can install the command line utility in a variety of ways. One of the easiest and most common is to use pipx:
pipx install era5_to_int
Overview
The era5_to_int utility converts ERA5 pressure-level netCDF files
to the WRF Pre-processing System (WPS) intermediate file format, permitting the
use of these data with either the Weather Research and Forecasting (WRF) model
or the Model for Prediction Across Scales - Atmosphere (MPAS-A).
The required command-line arguments are the path to the local files and the start and end datetimes of ERA5 files to convert. For example:
era5_to_int /root/path/to/era5/ 2024-05-01_00 2024-05-02_00
or if you're using a different time format, then you'll need to put quotes around the datetimes:
era5_to_int /root/path/to/era5/ "2024-05-01 00" "2024-05-02 00"
The directory structure must be the same as NCAR's ERA5 AWS S3 bucket. This is the recommended place to download the ERA5 data since it's free and fast.
Conversion of a range of datetimes is possible through the use of additional command-line arguments as described in the Usage section.
The following surface fields from the d633 datasets are handled by the script:
| Field | Dataset | Horiz. grid | Num. levels |
|---|---|---|---|
| SOILGEO | d633000 | 0.25-deg Lat-Lon | 1 |
| SP | d633006 | 0.25-deg Lat-Lon | 1 |
| MSL | d633000 | 0.25-deg Lat-Lon | 1 |
| VAR_2T | d633000 | 0.25-deg Lat-Lon | 1 |
| VAR_2D | d633000 | 0.25-deg Lat-Lon | 1 |
| VAR_10U | d633000 | 0.25-deg Lat-Lon | 1 |
| VAR_10V | d633000 | 0.25-deg Lat-Lon | 1 |
| RSN | d633000 | 0.25-deg Lat-Lon | 1 |
| SD | d633000 | 0.25-deg Lat-Lon | 1 |
| LSM | d633000 | 0.25-deg Lat-Lon | 1 |
| SSTK | d633000 | 0.25-deg Lat-Lon | 1 |
| SKT | d633000 | 0.25-deg Lat-Lon | 1 |
| SWVL1 | d633000 | 0.25-deg Lat-Lon | 1 |
| SWVL2 | d633000 | 0.25-deg Lat-Lon | 1 |
| SWVL3 | d633000 | 0.25-deg Lat-Lon | 1 |
| SWVL4 | d633000 | 0.25-deg Lat-Lon | 1 |
| STL1 | d633000 | 0.25-deg Lat-Lon | 1 |
| STL2 | d633000 | 0.25-deg Lat-Lon | 1 |
| STL3 | d633000 | 0.25-deg Lat-Lon | 1 |
| STL4 | d633000 | 0.25-deg Lat-Lon | 1 |
| CI | d633000 | 0.25-deg Lat-Lon | 1 |
When pressure-level (isobaric) ERA5 files are being processed, the following atmospheric fields are handled:
| Field | Dataset | Horiz. grid | Num. levels |
|---|---|---|---|
| Z | d633000 | 0.25-deg Lat-Lon | 37 |
| Q | d633000 | 0.25-deg Lat-Lon | 37 |
| T | d633000 | 0.25-deg Lat-Lon | 37 |
| U | d633000 | 0.25-deg Lat-Lon | 37 |
| V | d633000 | 0.25-deg Lat-Lon | 37 |
As fields are converted from netCDF to intermediate format, their names are also converted to match WPS and MPAS-A expectations.
Downloading and spatial clipping the ERA5 data
This command line utility isn't very helpful if you don't have the ERA5 data. There is a docker image made specifically to download the ERA5 data from the AWS S3 bucket maintained by NCAR. Then the container will clip the global files to a user-defined region to substantially shrink the files. It will save the clipped files to a directory or optionally upload it to another S3 system defined by the user.
You can find the github repo here: https://github.com/envlib/era5-download
It will download all of the files necessary to run the era5_to_int utility.
Usage
The required command-line arguments are the path to the local files and the start and end datetimes of ERA5 files to convert. For example:
era5_to_int /root/path/to/era5/ 2024-05-01_00 2024-05-02_00
Upon successful completion, an intermediate file with the prefix ERA5 is
created in the current working directory; for example, ERA5:2024-05-01_00.
By default, fields are converted at a six-hourly interval, and a different interval may be specified with the option -h. For example, to convert data at a three-hourly interval for the month of May 2024:
era5_to_int -h 3 /root/path/to/era5/ 2024-05-01_00 2024-05-02_00
Also by default, all variables required to run WRF are processed. If only a selection of variables is required, then use the --variables option:
era5_to_int --variables LANDSEA,SEAICE,SKINTEMP /root/path/to/era5/ 2024-05-01_00 2024-05-02_00
If you want all variables except a few, use --skip-vars (composes with --variables if both are given). Skipped variables do not need their ERA5 NetCDF files on disk, so this is useful when sourcing a field from elsewhere — for example, providing SST and sea ice from a higher-resolution product:
era5_to_int --skip-vars SST,SEAICE /root/path/to/era5/ 2024-05-01_00 2024-05-02_00
Usage help is provided by running the era5_to_int with the --help
argument.
Incomplete input
Since 0.5.0, every input is resolved before anything is written. For each time in the window and each variable to be
converted, era5_to_int finds the ERA5 file and the record holding that time. If any file is missing, any file lacks
a needed time, or any file cannot be read, it lists every problem on stderr (each file once, with the times it affects)
and exits 1 without writing an intermediate file.
Before 0.5.0, a missing file stopped the run mid-frame with the message on stdout, leaving a short last intermediate file, and a time absent from an existing file was silently written from the file's first record, that is, another time's data. The invariant files (land-sea mask, surface geopotential) hold a single 1979 record, which is used for every time by design.
Metadata
Release files for era5_to_int 0.5.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| era5_to_int-0.5.0.tar.gz | 26.8 kB | Without provenance | Details for era5_to_int-0.5.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| era5_to_int-0.5.0-py3-none-any.whl | Python 3 | none | any | Details for era5_to_int-0.5.0-py3-none-any.whl |
Total release size: 53.1 kB
Release files / era5_to_int-0.5.0.tar.gz
| Download URL | era5_to_int-0.5.0.tar.gz |
|---|---|
| Size | 26.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fd3fa08dc55f7d29820854ea25f8fa802c510f7b3bf2993a300c56bc8361943c
|
|
BLAKE2b-256 checksum How to use checksums |
f30a85a6d2b70c0c18c51d26d9138aa9a84dd4e7282b9d815f59d6fe9874eecb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.7
|
Release files / era5_to_int-0.5.0-py3-none-any.whl
| Download URL | era5_to_int-0.5.0-py3-none-any.whl |
|---|---|
| Size | 26.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3af864f34a056aa9442caa681ae4de9a1e070d2fec018e28f567bb64559481cc
|
|
BLAKE2b-256 checksum How to use checksums |
bbd4d2401f5aac027ece55bf34836e32a97f4b5177191883dc72250ec38290e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.7
|
Security for era5_to_int 0.5.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.