Skip to main content

madis-data

madis-data produces analysis-ready full-hour MADIS Mesonet and MADIS Metar datasets for arbitrary date ranges and operationally meaningful U.S. regions or individual stations as netCDF files.

Currently, madis-data operates with the following publicly available MADIS Mesonet and Metar observables:

  • 2 m temperature,
  • 2 m dewpoint temperature,
  • 10 m west-east and south-north wind speed,
  • solar radiation (Mesonet only).

The Jupyter notebook HowTo gives an overview of the functionality madis-data provides.

MADIS Mesonet and Metar data are provided by the National Centers for Environmental Prediction.

Installation

pip

pip install madis-data

Conda/Mamba

mamba install -c jan.kazil -c conda-forge madis-data

Jupyter

If the Conda/Mamba environment in which madis-data was installed is not yet available as a Jupyter kernel, install the Jupyter dependencies and register the environment as a kernel:

mamba install -c conda-forge jupyter_client jupyter_core notebook ipykernel

python -m ipykernel install --user --name "$CONDA_DEFAULT_ENV" --display-name "Python ($CONDA_DEFAULT_ENV)"

The kernel will then appear in Jupyter as Python (<environment-name>).

Overview

The package provides command-line tools that:

  • select stations by geography (an individual Mesonet or Metar station, a U.S. state, a Regional Transmission Organization/Independent System Operator regions, and the special region CONUS representing the contiguous United States),
  • download Mesonet or Metar observation files for a specified date range,
  • extract, preprocess, and save in netCDF files the observables
    • 2 m temperature,
    • 2 m dewpoint temperature,
    • 10 m west-east and south-north wind speed,
    • solar radiation (Mesonet only).
  • optionally removes the original MADIS files after successful preprocessing to reduce storage use,
  • constructs full-hourly time series for the supported observables,
  • and saves the spatially filtered observations and hourly time series in netCDF files.

It also generates a map showing the selected station locations.

Once preprocessed files with the above variables have been generated for a given date range, constructing full-hourly time series for that date range for individual stations, U.S. states, or the CONUS region no loner requires downloading the original Mesonet or Metar observations again.

Geospatial region selection is based on U.S. Energy Information Administration definitions of Regional Transmission Organization/Independent System Operator footprints and U.S. Census Bureau state and territory boundaries included with the package.

Workflow using command-line interface (CLI)

The CLI is exposed as "build-mesonet-dataset" and "build-metar-dataset" when installed. These executables

  1. download Mesonet or Metar surface observations data files, respectively, for a given time range from the MADIS archive,
  2. extract data of the following observables for stations in a given region:
    • 2 m temperature,
    • 2 m dewpoint temperature,
    • 10 m west-east and south-north wind speed,
    • solar radiation (Mesonet only).
  3. save the regional data in a netCDF file,
  4. interpolate the regional data to full-hourly time series,
  5. save the full-hourly time series for the stations in the region in a netCDF file,
  6. plot the stations in the region on a US map and save it as a PNG file.

Usage

build-mesonet-dataset START_YEAR START_MONTH START_DAY END_YEAR END_MONTH END_DAY REGION DATA_DIR \
    [-n N_JOBS] [--remove-original] [-r] [-v]
build-metar-dataset START_YEAR START_MONTH START_DAY END_YEAR END_MONTH END_DAY REGION DATA_DIR \
    [-n N_JOBS] [--remove-original] [-r] [-v]

Positional arguments

  • START_YEAR, START_MONTH, and START_DAY: Components of the first requested timestamp at 00:00 UTC
  • END_YEAR, END_MONTH, and END_DAY: Components of the last requested timestamp at 00:00 UTC
  • REGION: Region or station selector. Use a two-letter U.S. state or territory code, CONUS, an RTO/ISO code, or a MADIS Mesonet or Metar station identifier
  • DATA_DIR: Directory where downloaded data and generated outputs will be stored

Options

  • -n, --n N_JOBS: Number of parallel workers used for downloading, spatial extraction, and interpolation. Values greater than 1 can accelerate processing but may increase memory use or the risk of network errors.
  • --remove-original: Remove downloaded original MADIS files after successful preprocessing.
  • -r, --refresh: Download files even when corresponding local files already exist.
  • -v, --verbose: Print detailed progress information.

The command downloads one additional day before and after the requested interval so observations can bracket its boundary hours during interpolation. It creates a NetCDF file containing the selected record-oriented observations, a map of the selected station locations, and a NetCDF file containing the merged full-hourly UTC time series.

Examples

# Show usage information:
build-mesonet-dataset --help

# Download and process MADIS Mesonet data for station SMPC2 from 00:00 UTC on January 1, 2026,
# through 00:00 UTC on January 31, 2026, and save the results in /path/to/data:
build-mesonet-dataset -v 2026 1 1 2026 1 31 SMPC2 /path/to/data

# Build MADIS Metar datasets for Colorado using 16 parallel workers and remove source files after preprocessing:
build-metar-dataset -v 2026 1 1 2026 2 1 CO /path/to/data -n 16 --remove-original

Example scripts

The Bash scripts

./scripts/Download_MADIS_Mesonet.CO.2026-01.sh
./scripts/Download_MADIS_Metar.CO.2026-01.sh

can be executed from within the directory ./scripts. They download MADIS Mesonet or Metar surface observation data files from the MADIS archive, respectively, and build full-hourly datasets for Colorado for January 2026, in the directory ./data.

Requirements

The madis-data package must be installed in the Python environment in which the scripts are called.

Workflow using API

The madis-data workflow is documented in the HowTo Jupyter notebook. The notebook performs the following tasks:

  1. download Metar surface observations data files for January 2026 from the MADIS archive,
  2. extract January 2026 data for stations in the state of Colorado of the following observables:
    • 2 m temperature,
    • 2 m dewpoint temperature,
    • 10 m west-east and south-north wind speed,
  3. save the extracted data in a s netCDF file,
  4. interpolate the data to full-hourly time series,
  5. save the full-hourly time series in a netCDF file,
  6. plot the original and the full-hourly interpolated time series at an individual station over select time intervals,
  7. plot the stations in the state of Colorado on a US map.

The analogous workflow applies to MADIS Metar data.

Requirements

The notebook requires a Jupyter kernel being available in the Python environment in which madis-data has been installed (Section "Jupyter").

Notes

MADIS Mesonet data files represent > 30000 stations. Downloading the hourly files for extended periods of time can take hours to days. Creating full-hourly time series requires sufficient computing power and RAM. E.g., creating the full-hourly time series for stations in Texas, for the period 2021-2025, takes on a machine with 128 GB RAM and 8 cores approximately 4 h. Downloading and processing MADIS Metar data files takes significantly less time owing to the lower number of Metar stations.

Development

Code Quality and Testing Commands

  • make fmt - Runs ruff format, which reformats Python files according to the style rules in pyproject.toml.
  • make lint - Runs ruff check --fix, which lints the code and auto-fixes what it can.
  • make check - Runs formatting and linting.
  • make type - Currently disabled. Intended to run mypy using the settings in pyproject.toml.
  • make test - Runs pytest with the test settings configured in pyproject.toml.

Disclaimers

This software uses publicly available NOAA National Centers for Environmental Prediction (NCEP) data that are subject to their terms of use, but is neither affiliated, endorsed, or certified by NOAA or NCEP.

This software uses publicly available U.S. Census Bureau data that are subject to their terms of use, but is neither affiliated, endorsed, or certified by the U.S. Census Bureau.

This software uses publicly available U.S. Energy Information Administration data that are subject to their terms of use, but is neither affiliated, endorsed, or certified by the U.S. Energy Information Administration.

This software uses publicly available University Corporation for Atmospheric Research (UCAR) data that are subject to their terms of use, but is neither affiliated, endorsed, or certified by UCAR.

Author

Jan Kazil - jan.kazil.dev@gmail.com - github.com/jankazil/madis-data

License

BSD-3-Clause

Metadata

Release files for madis-data 0.2.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for madis-data 0.2.10
File Size Uploaded Provenance Actions
madis_data-0.2.10.tar.gz 22.4 MB Without provenance Details for madis_data-0.2.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for madis-data 0.2.10
File Interpreter ABI Platform Actions
madis_data-0.2.10-py3-none-any.whl Python 3 none any Details for madis_data-0.2.10-py3-none-any.whl

Total release size: 44.9 MB

Release files / madis_data-0.2.10.tar.gz

Download URL madis_data-0.2.10.tar.gz
Size 22.4 MB
Tags Source
SHA-256 checksum
How to use checksums
230310d372099862d5c75a3c385cb13174dfb1bc2729f4ac43498d28410dfc2e
BLAKE2b-256 checksum
How to use checksums
8f3e9c13196914d3fc1f5cf20ed3eea4e7419347a079bb2334073b785789c2bb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Release files / madis_data-0.2.10-py3-none-any.whl

Download URL madis_data-0.2.10-py3-none-any.whl
Size 22.5 MB
Tags Python 3
SHA-256 checksum
How to use checksums
e69d79dfcc452e3ab6bfc779c1a6ad81960a1444b39f02fec729a6b588b36041
BLAKE2b-256 checksum
How to use checksums
3cd7b3d9cdd716f8d477a855f093f0d056627ff3c22e273ffd955e2450bee102
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.13

Security for madis-data 0.2.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page