Yet another NetCDF checker
Project description
This command-line program checks that a NetCDF file conforms to a pre-defined specification. It checks if the file contains the specified dimensions and variables; that the dimensions are of the correct size; that the variables have values within range; and that the variables have certain attributes.
Example use
python yanc.py\
--ncfile http://thredds.met.no/thredds/dodsC/meps25files/meps_det_pp_2_5km_latest.nc \
--template templates/meps_det_pp_2_5km.yml
Template
Yanc checks the NetCDF file against specifications in a template file. The template understands the following structure:
dimensions:
- name: time
length: 15
- name: x
min_length: 10
max_length: 20
variables:
- name: air_temperature
min: 200
max: 400
units: K
'% missing': 10
missing_timesteps: [0,1,2]
All attributes are optional, and yanc will not check dimensions/variables/min/max/etc when not specified. Dimensions can be checked for an exact length (by using the work ‘length’) or a range of lengths (inclusively, by using ‘min_length’ and ‘max_length’).
Neither ‘dimensions’ nor ‘variables’ are required to be present in the file.
Installation
The easiest way to install yanc is using pip:
pip install metyanc
To install from source, install like this:
pip install -r requirements.txt
python setup.py install
Return codes
The program returns 0 if the file is valid. Otherwise a non-zero code is returned. If the template file is missing or contains invalid YAML, or if the NetCDF file does not exist or is invalid NetCDF, then a non-zero code is returned.
Missing values
By default, variables are not allowed to have missing values. To allow a missing values, the ‘% missing’ can be added, which means the test will fail if there the percentage of missing values exceeds the allowed value. Some variables (such as precipitation_amount) will have have missing values on the first timestep. Data in timesteps specified by missing_timesteps will not count towards the percentage missing.
For example, the following requires the precipitation_amount variable to have no missing values except for the first time step:
variables:
- name: precipitation_amount
units: m
'% missing': 0
missing_timesteps: 0
missing_timesteps can also be an array like this:
variables:
- name: precipitation_amount
units: m
'% missing': 0
missing_timesteps: [0, 1, 2]
Testing
Running “make test” runs some quick unittests. Running “make testop” checks that the current operational files conform to specifications. This can take 10-15 minutes.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.