Subset National Water Model (NWM) NetCDF Ver1.1
Project description
# subset_nwm_netcdf
Note that the deployment schedule of new National Water Model (NWM) Ver1.1 got delayed until May 4th 2017 according to its official notice: http://www.nws.noaa.gov/os/notification/scn17-41natl_water_model.htm
NWM's daily output is about 400GB worth of NetCDF files that contain meteorological and hydrologic data/forecasts covers the whole U.S.
This python library enables users to subset NWM Ver1.1 NetCDF using a polygon that represents a region of interest, which can significantly reduce data storage size and speed up regional data access.
More about NWM: http://water.noaa.gov/about/nwm
## Version naming convention:
"1.1.2", where "1.1" means NWM Ver1.1 and "2" is the version of this library.
## Workflow:
user-provided polygon --> Spatial Query module* --> stream comids & reservoir comids & grid cells --> Subset & Merge module --> regional NetCDF files
*: Spatial Query is optional if user directly provides stream comids & reservoir comids & grid cells
## Usage
See demo.py in source
The copy of the resulting NetCDF files of a huc_12 watershed (comid: 160102040504) for date 20170327 can be found at https://www.hydroshare.org/resource/734533a9e08c494aa28d2d0e688e2c06/
## Limitations
1)Currently this library DOES NOT support subsetting the following files:
1-1) "tm01" and "tm02" of each time stamp in analysis_assim model configuration
1-2) the "terrain" files in all model configurations
2) Names of source NWM NetCDF files should be kept unchanged and stored in default folder structure
## Environment and Dependencies:
Preliminary functional testings passed with Python 2.7.12 x64 on Windows 7 x64 and Ubuntu 16.04 x64.
### Spatial Query module:
1) fiona @ https://pypi.python.org/pypi/Fiona
2) shapely @ https://pypi.python.org/pypi/Shapely/
3) pysqlite with mod_spatialite extension @ https://pypi.python.org/pypi/pysqlite/ and https://www.gaia-gis.it/fossil/libspatialite/wiki?name=mod_spatialite
4) NWM sqlite geodatabase (4.5GB split zip files) @ https://www.hydroshare.org/resource/95410260015a4fd1858a3ad3c4aa7f17/
### Subset & Merge module
1) NetCDF utilities (shell commands) @ https://www.unidata.ucar.edu/downloads/netcdf/index.jsp
2) NCO (shell commands) @ http://nco.sourceforge.net/
3) numpy @ https://pypi.python.org/pypi/numpy
4) netCDF4 python lib @ https://pypi.python.org/pypi/netCDF4
5) sed (shell command)
Implementation inspired by:
https://github.com/shawncrawley/subset_nwm_data_scripts
https://github.com/twhiteaker/pynwm
Note that the deployment schedule of new National Water Model (NWM) Ver1.1 got delayed until May 4th 2017 according to its official notice: http://www.nws.noaa.gov/os/notification/scn17-41natl_water_model.htm
NWM's daily output is about 400GB worth of NetCDF files that contain meteorological and hydrologic data/forecasts covers the whole U.S.
This python library enables users to subset NWM Ver1.1 NetCDF using a polygon that represents a region of interest, which can significantly reduce data storage size and speed up regional data access.
More about NWM: http://water.noaa.gov/about/nwm
## Version naming convention:
"1.1.2", where "1.1" means NWM Ver1.1 and "2" is the version of this library.
## Workflow:
user-provided polygon --> Spatial Query module* --> stream comids & reservoir comids & grid cells --> Subset & Merge module --> regional NetCDF files
*: Spatial Query is optional if user directly provides stream comids & reservoir comids & grid cells
## Usage
See demo.py in source
The copy of the resulting NetCDF files of a huc_12 watershed (comid: 160102040504) for date 20170327 can be found at https://www.hydroshare.org/resource/734533a9e08c494aa28d2d0e688e2c06/
## Limitations
1)Currently this library DOES NOT support subsetting the following files:
1-1) "tm01" and "tm02" of each time stamp in analysis_assim model configuration
1-2) the "terrain" files in all model configurations
2) Names of source NWM NetCDF files should be kept unchanged and stored in default folder structure
## Environment and Dependencies:
Preliminary functional testings passed with Python 2.7.12 x64 on Windows 7 x64 and Ubuntu 16.04 x64.
### Spatial Query module:
1) fiona @ https://pypi.python.org/pypi/Fiona
2) shapely @ https://pypi.python.org/pypi/Shapely/
3) pysqlite with mod_spatialite extension @ https://pypi.python.org/pypi/pysqlite/ and https://www.gaia-gis.it/fossil/libspatialite/wiki?name=mod_spatialite
4) NWM sqlite geodatabase (4.5GB split zip files) @ https://www.hydroshare.org/resource/95410260015a4fd1858a3ad3c4aa7f17/
### Subset & Merge module
1) NetCDF utilities (shell commands) @ https://www.unidata.ucar.edu/downloads/netcdf/index.jsp
2) NCO (shell commands) @ http://nco.sourceforge.net/
3) numpy @ https://pypi.python.org/pypi/numpy
4) netCDF4 python lib @ https://pypi.python.org/pypi/netCDF4
5) sed (shell command)
Implementation inspired by:
https://github.com/shawncrawley/subset_nwm_data_scripts
https://github.com/twhiteaker/pynwm
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file subset_nwm_netcdf-1.1.2.tar.gz
.
File metadata
- Download URL: subset_nwm_netcdf-1.1.2.tar.gz
- Upload date:
- Size: 1.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
0847daf5d13c43f0cdc5f87eb9073e9776577dc74ac40180716700749afdbebf
|
|
MD5 |
a2d6cbf0435a227a13548c621c94aa5e
|
|
BLAKE2b-256 |
2a78d02da399838bf8527f2c6b4e8fc211827991eab59ec5b964975fdcf24c53
|