ugrid-checks 0.2.0

Check netcdf files for conformance with UGRID specifications

ugrid-checks

Version : 0.2.0

A utility to check netcdf files to the UGRID specification.

It tests files against the UGRID conformance rules, and can also produce a summary of the mesh content of a file.

• Installation
• Command Line : checking
  • Basic usage
  • Controlling checks
• Command Line : structure analysis
• List of Conformance Rules and codes
• Limitations
• Python API
• Runtime Requirements
• Known Issues
• Raising Issues
• Changelog

Installation

Available on PyPI and conda-forge.

To install:

> pip install ugrid-checks

or

> conda create -n ugrid_checker ugrid-checks

Command Line : checking

$ ugrid-checker -h
usage: ugrid-checker [-h] [-q] [-i IGNORE] [-e] [-d MAX_DATASIZE] [-s] [--nonmesh] [-v] [file]

Check a netcdf-CF file for conformance to the UGRID conventions.

positional arguments:
  file                  File to check.

optional arguments:
  -h, --help            show this help message and exit
  -q, --quiet           don't print a checking report if there were no problems
  -i IGNORE, --ignore IGNORE
                        a list of errorcodes to ignore.
  -e, --errorsonly      ignore all warnings ("Axxx"= advise codes), and only report errors ("Rxxx"= require codes).
  -d MAX_DATASIZE, --max-datasize MAX_DATASIZE
                        maximum array-size, above which variable-data checks are skipped. Default=200.0 (Mb).
  -s, --summary         print a summary of UGRID mesh information found in the file
  --nonmesh             include a list of non-mesh variables in the summary
  -v, --version         print version information

$ 

Basic usage

$ ugrid-checker data_C4.nc

UGRID conformance checks complete.

No problems found.

Done.
$

$ ugrid-checker data_C4_warn_error.nc 

UGRID conformance checks complete.

List of checker messages :
  *** FAIL R106 : Mesh variable "topology" attribute 'face_coordinates' refers to a variable "longitude", but there is no such variable in the dataset.
  *** FAIL R108 : Mesh variable "topology" has face_coordinates="latitude longitude", which is not a list of variables in the dataset.
  ... WARN A304 : Mesh connectivity variable "edge_nodes" of mesh "topology" has a '_FillValue' attribute, which should not be present on a "edge_node_connectivity" connectivity.

Total of 3 problems logged :
  2 Rxxx requirement failures
  1 Axxx advisory recommendation warnings

Done.
$

Controlling checks

The -e / --errorsonly option checks only against the requirements (aka "errors"), and skips the recommendations (aka "warnings").

The -i / --ignore option skips particular checks according to their Axxx/Rxxx codes.
See List of codes.

Example:

$ ugrid-checker data_C4_warn_error.nc --errorsonly --ignore r106,r108

Ignoring codes:
  R106, R108

UGRID conformance checks complete.

No problems found.

Done.
$

Command line : structure analysis

The -s / --summary option prints a summary of a file's mesh content.

$ ugrid-checker data_C4.nc --summary --quiet

File mesh structure
-------------------
Meshes
    "topology"
        node("num_node")
            coordinates : "node_lat", "node_lon"
        face("dim0")
            face_node_connectivity : "face_nodes"
            coordinates : "latitude", "longitude"

Mesh Data Variables
    "sample_data"
        mesh : "topology"
        location : "face"

$

List of Conformance Rules and codes

All the error/warning codes used are defined in the UGRID conformance rules. Each has an identifying code : "Rxxx" for requirements or "Axxx" for advisory rules.

See the list here : UGRID Draft Conformance Rules

Note : these are currently only available in this preliminary draft version, not yet accepted into the UGRID spec.

Limitations

No 3-D / Volume support

The draft conformance rules do not currently cover 3-d meshes, so neither does the checker code. There is also an ongoing discussion whether it is needed to "fix" this feature, or even to remove it in a future release of UGRID. See this developer discusssion.
Alternatively, this can be added in future.

Python API

The checker is provided as an importable module "ugrid_checks".

Running the module directly calls the command-line interface,
e.g. $ python -m ugrid_checks file.nc is the same as $ ugrid-checker file.nc.

Within Python, the module can be used like this :

>>> from ugrid_checks.check import check_dataset
>>> print(check_dataset.__doc__)

    Run UGRID conformance checks on a file.

    Optionally print a result summary.
    Optionally ignore errors below a logging level.
    Returns a checker object with a file analysis and checking log records.

    Parameters
    ----------
    file : string, Path or :class:`NcFileSummary`
        path to, or representation of a netcdf input file
    print_summary : bool, default=True
        print a results summary at the end
    omit_advisories : bool, default False
        If set, log only 'requirements' Rxxx statements, and ignore the
        advisory 'Axxx' ones.
    ignore_codes : list(str) or None, default None
        A list of error codes to ignore.
    max_data_mb : float, default 200.0
        A rough size threshold (in Mb), beyond which we will skip data checks.
        Set to 0 for no checking, or -1 for no size limit.

    Returns
    -------
    checker : Checker
        A checker for the file.
    
>>>
>>> checker = check_dataset('data_C4_warn.nc', print_summary=False)
>>> 
>>> type(checker)
<class 'ugrid_checks.check.Checker'>
>>>
>>> print(checker.checking_report())
UGRID conformance checks complete.

List of checker messages :
  ... WARN A304 : Mesh connectivity variable "edge_nodes" of mesh "topology" has a '_FillValue' attribute, which should not be present on a "edge_node_connectivity" connectivity.

Total of 1 problems logged :
  0 Rxxx requirement failures
  1 Axxx advisory recommendation warnings

Done.
>>> 
>>> print(checker.structure_report())
Meshes
    "topology"
        node("num_node")
            coordinates : "node_lat", "node_lon"
        face("dim0")
            face_node_connectivity : "face_nodes"
            coordinates : "latitude", "longitude"

Mesh Data Variables
    "sample_data"
        mesh : "topology"
        location : "face"
>>>

For the time being, there is no built API documentation : Please refer to code docstrings for more detail.

Runtime Requirements

• Python >= 3.7
• netCDF4

Known Issues

• none known

Raising Issues

Please create an issue in : https://github.com/pp-mo/ugrid-checks

Changelog

• TODO: move to separate file ?

• v0.2.0

  • #45 implemented all checks on variable data content, subject to a datasize threshold
  • #48 relaxed the requirements for a valid variable name

• v0.1.2

  • release 2022-02-27
    • #36 fix to A304
      allow face-node-connectivity to have a _FillValue, as this is used for flexible 2d meshes.

• v0.1.1

  • release 2022-02-24
    • #23 structure-report bugfixes :
      • report optional connectivities, which were previously missing
      • report location-index-sets, which previously used to error
    • #26 corrected statement codes :
      Some statements in the range R109-R112 were being reported with the wrong statement code

• v0.1.0

  • release 2022-02-09