The NOAA Big Data Program has made weather data more accessible than ever before. Herbie is a python package that downloads recent and archived numerical weather prediction (NWP) model output from different cloud archive sources. Herbie helps you discover and download High Resolution Rapid Refresh (HRRR), Rapid Refresh (RAP), Global Forecast System (GFS), National Blend of Models (NBM), and Rapid Refresh Forecast System - Prototype (RRFS). NWP data is usually in GRIB2 format and can be read with xarray/cfgrib.
📔 Herbie Documentation
Install
Requires cURL and Python 3.8+ with requests, numpy, pandas, xarray, and cfgrib. Optional packages are matplotlib, cartopy, and Carpenter Workshop.
pip install herbie-data
or
pip install git+https://github.com/blaylockbk/Herbie.git
or, create the provided conda environment.
Capabilities
- Search different data sources for model output.
- Download full GRIB2 files
- Download subset GRIB2 files (by grib field)
- Read data with xarray
- Plot data with Cartopy (very early development)
from herbie.archive import Herbie
# Herbie object for the HRRR model 6-hr surface forecast product
H = Herbie('2021-01-01 12:00',
model='hrrr',
product='sfc',
fxx=6)
# Download the full GRIB2 file
H.download()
# Download a subset, like all fields at 500 mb
H.download(":500 mb")
# Read subset with xarray, like 2-m temperature.
H.xarray("TMP:2 m")
Data Sources
Herbie downloads model data from the following sources, but can be extended to include others:
- NOMADS
- Big Data Program Partners (AWS, Google, Azure)
- University of Utah CHPC Pando archive
History
During my PhD at the University of Utah, I created, at the time, the only publicly-accessible archive of HRRR data. In the later half of 2020, this data was made available through the NOAA Big Data Program. This package organizes and expands my original download scripts into a more coherent package with the ability to download HRRR and RAP model data from different data sources. It will continue to evolve at my own leisure.
I originally released this package under the name "HRRR-B" because it only dealt with the HRRR data set, but I have addeed ability to download RAP data. Thus, it was rebranded with the name "Herbie" as a model download assistant. For now, it is still called "hrrrb" on PyPI because "herbie" is already taken. Maybe someday, with some time and an enticing reason, I'll add additional download capabilities.
Alternative Download Tools
As an alternative you can use rclone to download files from AWS or GCP. I quite like rclone. Here is a short rclone tutorial
Thanks for using Herbie, and Happy Racing 🏎🏁
- Brian
👨🏻💻 Contributing Guidelines
💬 GitHub Discussions
🚑 GitHub Issues
🌐 Personal Webpage
🌐 University of Utah HRRR archive
✒ Pando HRRR Archive citation:
Blaylock B., J. Horel and S. Liston, 2017: Cloud Archiving and Data Mining of High Resolution Rapid Refresh Model Output. Computers and Geosciences. 109, 43-50. https://doi.org/10.1016/j.cageo.2017.08.005
P.S. If you like Herbie, check out my GOES-2-go package to download GOES-East/West data and SynopticPy to download mesonet data from the Synoptic API.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file herbie-data-0.0.6.tar.gz.
File metadata
- Download URL: herbie-data-0.0.6.tar.gz
- Upload date:
- Size: 40.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/0.0.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
760c4584031c327292a8423d78a45067a79f3f6b90fefb1548c5e269f4047f60
|
|
| MD5 |
fc51426308deaa7bf8cd61302ae8e6e7
|
|
| BLAKE2b-256 |
244dec4294d321e54ddeab6a65a097ea5ce77d49a0e1b5b290c7024279eee417
|
File details
Details for the file herbie_data-0.0.6-py3-none-any.whl.
File metadata
- Download URL: herbie_data-0.0.6-py3-none-any.whl
- Upload date:
- Size: 45.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/0.0.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7ec6b3bea414c2256e448eb1b77d411461a48d3beb999562862a683a13d15fd6
|
|
| MD5 |
3658ed02565b4dea7a4a5d885f0adceb
|
|
| BLAKE2b-256 |
33c5fb15ab269f9fad9ecd17a1524511f1511aa4eb82f517bf1bc6f527514048
|
