gamut-format 1.0.0

An open file format for parametric simulation studies,
with libraries for Python, MATLAB, Julia, C++ and C.

Gamut is a file format for parametric simulation studies: the hundreds or thousands of CFD or FEA runs behind a design sweep, an aerodynamic database or a surrogate model. One file holds the run table, each run's inputs, results and status, and, when the runs have a mesh, the mesh and every field on it, all with units. Python, MATLAB, Julia and C++ read and write it with the same calls. Files are HDF5 laid out as netCDF-4, so h5py, xarray and the netCDF tools open them too, and a published specification says what every array in them means.

Example

Nine runs of potential flow past a cylinder on one mesh: eight written as a sweep, a ninth added after reopening the file, then the pressure at one node read back from every run. points, cells and solve stand for your mesh and your solver, and the complete programs are in examples/.

Python

with gamut.create("cylinder.gamut", writer="example sweep", mesh="shared", per="fixed",
                   coordinates={"units": "m", "axes": 2},
                   parameters={"gamma": {"units": "m2 s-1", "role": "design"}},
                   fields={"cp": {"units": "1", "at": "node"}}) as study:
    study.set_mesh(points, cells)
    for gamma in np.linspace(0.0, 3.5, 8):
        study.add_run({"gamma": gamma}, fields={"cp": solve(gamma)})

with gamut.open("cylinder.gamut", mode="a") as study:
    study.add_run({"gamma": 4.0}, fields={"cp": solve(4.0)})

with gamut.open("cylinder.gamut") as study:
    cp = study.get_field("cp", nodes=[12])                 # runs x nodes

MATLAB

study = gamut.create('cylinder.gamut', 'writer', 'example sweep', 'mesh', 'shared', 'per', 'fixed', ...
    'coordinates', struct('units', 'm', 'axes', 2), ...
    'parameters', struct('gamma', struct('units', 'm2 s-1', 'role', 'design')), ...
    'fields', struct('cp', struct('units', '1', 'at', 'node')));
study.setMesh(points, cells);
for gamma = linspace(0, 3.5, 8)
    study.addRun(struct('gamma', gamma), 'fields', struct('cp', solve(gamma)));
end
study.close();

study = gamut.open('cylinder.gamut', 'mode', 'a');
study.addRun(struct('gamma', 4), 'fields', struct('cp', solve(4)));
study.close();

study = gamut.open('cylinder.gamut');
cp = study.getField('cp', 'nodes', 13);                         % runs x nodes, from 1
study.close();

Julia

Gamut.create("cylinder.gamut"; writer="example sweep", mesh="shared", per="fixed",
              coordinates=(units="m", axes=2),
              parameters=Dict("gamma" => (units="m2 s-1", role="design")),
              fields=Dict("cp" => (units="1", at="node"))) do study
    Gamut.set_mesh!(study; points, cells)
    for gamma in range(0, 3.5; length=8)
        Gamut.add_run!(study, Dict("gamma" => gamma); fields=Dict("cp" => solve(gamma)))
    end
end

Gamut.open("cylinder.gamut"; mode="a") do study
    Gamut.add_run!(study, Dict("gamma" => 4.0); fields=Dict("cp" => solve(4.0)))
end

cp = Gamut.open("cylinder.gamut") do study
    Gamut.get_field(study, "cp"; nodes=[13])                 # runs × nodes, from 1
end

C++

auto study = gamut::create("cylinder.gamut", "example sweep", "shared", "fixed");
study.declare_coordinates("m", 2);
study.declare_parameter("gamma", "design", "m2 s-1");
study.declare_field("cp", "node", "1");
study.set_mesh(points, cells);
for (double gamma = 0.0; gamma < 3.75; gamma += 0.5)
  study.add_run({{"gamma", gamma}}, {{"cp", solve(gamma)}});
study.close();

study = gamut::open("cylinder.gamut", "a");
study.add_run({{"gamma", 4.0}}, {{"cp", solve(4.0)}});
study.close();

study = gamut::open("cylinder.gamut");
auto cp = study.get_field("cp", gamut::all, {12});    // runs x nodes

All four print the same values:

cp at the top of the cylinder, run by run: -3.00 -2.69 -2.39 -2.10 -1.83 -1.57 -1.32 -1.08 -0.86

MATLAB and Julia count positions from 1, so node 12 in Python and C++ is node 13 in MATLAB and Julia.

What you can do with it

Performance

Gamut is about the size of netCDF-4 and Zarr, a little smaller than VTKHDF and HDF5 written a run at a time, and a sixth of the VTK files. Choosing runs by condition and reading a result of each takes about a millisecond, as fast as a Parquet file of the run table alone. Opening a file costs Gamut about 0.9 ms, so HDF5 and VTKHDF read straight with h5py fetch one small run faster, and uncompressed NumPy arrays, at 2.3 times the size, read fastest of all. Gamut's chunks are laid out for reading whole runs, so one node across every run takes 127 ms, slower than Zarr or netCDF-4. HDF5 decompresses on one core; with threads turned on, Gamut fetches the compressed chunks of a whole-run or contiguous read through HDF5 and decompresses them in parallel, and four threads bring that read to 39 ms. On the AirfRANS study queried in the picture at the top, with meshes of about 176,000 nodes, one run reads in 15 ms against 124 ms from its VTK file.

_{Apple M3, 8 GB, macOS 27, files in the operating system's cache; medians of five reads, each opening the file. The benchmarks page has every number, the method and the scripts to repeat it.}

Install

The Python wheels for macOS, Linux and Windows contain the core with HDF5 linked in. MATLAB and Julia use the same core: the wheel's (gamut.library_path() prints where it is) or one built with CMake. Installing has the details for each language.

Documentation

The guide has one short page per concept, each running one script from examples/. The specification defines the format for anyone writing another implementation, and how it compares sets Gamut beside VTK, CGNS, Exodus, netCDF-CF and Zarr.

Before version 1 the format was called mestra. Its files (mestra/0) are refused with their version named; its 0.1 releases stay on PyPI under that name.

License

Apache 2.0. Made by Bijective Systems. If you use Gamut in published work, please cite it (CITATION.cff).