Skip to main content

ERA5 data pipeline for generating tropospheric refractivity datasets

Project description

GeoSol Research Logo

Tropo Data (Troposphere Data Processing)

ERA5 data download and processing tools for generating tropospheric refractivity datasets used by gri-tropo. Requires Python 3.12+.

Overview

This package provides scripts for:

  • Downloading ERA5 reanalysis data from the Copernicus Climate Data Store (CDS)
  • Processing surface data to calculate surface refractivity (n_sur)
  • Processing pressure-level data to calculate reference heights (ref_ht)

The output NPZ files are consumed by the gri-tropo library for tropospheric correction calculations.

Installation

# Clone and setup
git clone https://gitlab.com/geosol-foss/python/gri-tropo-data.git
cd gri-tropo-data
uv sync

# Activate environment
source .venv/bin/activate

Usage

1. Download ERA5 Data

Download ERA5 surface and pressure-level data for a specific date:

python -m gri_tropo_data.download_era5 2025 03 15

Output:

  • ./data/era5_raw/era5_surface_2025_03_15.nc - Surface meteorological data
  • ./data/era5_raw/era5_pressure_2025_03_15.nc - Pressure-level data

Note: Requires CDS API credentials. Register at the Copernicus Climate Data Store and configure your ~/.cdsapirc file with your UID and API key. See setup instructions in the script header.

2. Process Surface Refractivity Data

Process all unprocessed ERA5 surface data to generate n_sur files:

python -m gri_tropo_data.update_nsur

Output:

  • ./data/n_sur/n_sur_2025_03_15.npz - Surface refractivity data

3. Process Reference Height Data

Process all unprocessed ERA5 pressure-level data to generate ref_ht files:

python -m gri_tropo_data.update_refht

Output:

  • ./data/ref_ht/ref_ht_2025_03_15.npz - Reference height data

Data Formats

Downloaded ERA5 Data

Surface Data (Single-Level):

  • Variables: 2m temperature, surface pressure, 2m dewpoint temperature
  • Temporal resolution: Hourly (24 snapshots per day)
  • Spatial resolution: 1 x 1 degree global grid

Pressure-Level Data:

  • Variables: Temperature, geopotential, specific humidity
  • Pressure levels: 1000, 925, 850, 700, 500, 300 hPa
  • Temporal resolution: Hourly (24 snapshots per day)
  • Spatial resolution: 1 x 1 degree global grid

Processed NPZ Files

n_sur files - Shape: (181, 360, 24) for (latitude, longitude, hour)

  • Surface refractivity in N-units
  • Hourly data for a single day

ref_ht files - Shape: (181, 360) for (latitude, longitude)

  • Reference height in meters (scale height of exponential refractivity decay)
  • Daily average data

Indexing scheme:

  • Latitudes: indices 0-90 for 0 to 90 degrees N, indices 91-180 for -90 to -1 degrees S
  • Longitudes: indices 0-359 for 0 to 359 degrees E
  • Hours (n_sur only): indices 0-23 for UTC hours

See detailed format documentation for more information.

Configuration

All scripts default to ./data/ for input/output but can be configured by modifying the paths in the scripts.

Other Projects

Current list of other GRI FOSS Projects we are building and maintaining.

License

MIT License. See LICENSE for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gri_tropo_data-0.2.3.tar.gz (92.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

gri_tropo_data-0.2.3-py3-none-any.whl (25.2 kB view details)

Uploaded Python 3

File details

Details for the file gri_tropo_data-0.2.3.tar.gz.

File metadata

  • Download URL: gri_tropo_data-0.2.3.tar.gz
  • Upload date:
  • Size: 92.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for gri_tropo_data-0.2.3.tar.gz
Algorithm Hash digest
SHA256 c85a70509ab6994687148cb06d16ed4444e2b3b99d4f2e005562d691facdba73
MD5 4fc1c66d3d3c42ce5e9ae7694a16af8c
BLAKE2b-256 2b26836802fd516fb05a6ab752302d05e35d566b22e18941fc5b4ca7c2a1da1a

See more details on using hashes here.

File details

Details for the file gri_tropo_data-0.2.3-py3-none-any.whl.

File metadata

  • Download URL: gri_tropo_data-0.2.3-py3-none-any.whl
  • Upload date:
  • Size: 25.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Debian GNU/Linux","version":"13","id":"trixie","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for gri_tropo_data-0.2.3-py3-none-any.whl
Algorithm Hash digest
SHA256 377e6dc2d30d9638ecb4388c40a6f9ff2151e996d3987ea962a68b987cbd7b1a
MD5 ad129482de420dc8e12a1d937dafea75
BLAKE2b-256 39df15b8c852ea46e9609468c03f3df15b1b96859bce806fc67a2f1475fd8671

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page