dales-tile-combiner 0.1.0

Combine DALES MPI-split NetCDF tiles into an HDF5 virtual dataset (VDS).

DALES tile combiner

This script combines DALES MPI-split NetCDF (.nc) files into a single HDF5 Virtual Dataset (VDS) file. This HDF5 file is made in such a way as to appear like a netCDF file, so most readers I've tried don't even know it's a VDS.

The resultant file can be read with any of your usual netCDF tools, and all attributes in the original files should be preserved.

Installation

Install dependencies via requirements.txt:

pip install -r requirements.txt

Usage

python dales_tile_combiner.py <data_directory>

given the directory

<data_directory>
    fielddump.000.000.001.nc
    fielddump.000.001.001.nc
    fielddump.000.002.001.nc
    fielddump.000.003.001.nc

yields

fielddump.nc # is an in-file relative symbolic link to <data_directory>
<data_directory> 
    fielddump.000.000.001.nc
    fielddump.000.001.001.nc
    fielddump.000.002.001.nc
    fielddump.000.003.001.nc

yields The script automatically detects known file types (fielddump, cape, surfcross, slrbcross, crossxy, crossxz, crossyz) and generates combined VDS files in the current directory (e.g., fielddump.nc).

Crosssections

Crosssections made with modcrossection (crossxy, crossxz, crossyz) are all combined into a single file. They are combined along dimension (zt/zm, yt/ym, xt/xm) respectively and given coordinate (zt/zm, yt/ym, xt/xm) corresponding to their INDEX. (not their actual coordinate as that information is not available within the crosssection file)

Limitations

• There is no support yet for LSM crosssections like lsmcrossyz and lxmcrossxz.
• The resultant file does not contain the actual data, merely (relative) links to the data files. Moving the datafiles thus breaks the VDS file.

Bugs?

Please raise an issue and let me know if I can do anything.