Recast WOOF

Weather Observations and Open Forecasting: a GPU-native weather model you run on your own card.

This is a fork. WOOF is written and maintained by Recast Systems at github.com/recastsystems/woof, which is the original and main code, and the place to start. This fork, github.com/jacobbieker/woof, merges each upstream release and adds the work listed under This fork below, most notably a backend that runs on Apple-silicon GPUs. On PyPI, Recast's release is recast-woof; this fork's Apple-silicon build is metal-woof, which is unofficial and not supported by Recast Systems.

WOOF is a regional weather model with WRF-class physics that runs on one NVIDIA GPU, with its own preprocessing, data assimilation and plots. One install carries two models:

Preview means the model is in the package, documented and tested on CPU, and its command says preview each time it runs; its config and checkpoint formats may change in a minor release. The major-version promise covers the regional model.

Production means the regional commands and their config and checkpoint formats are covered by that promise. It is not a statement about forecast accuracy; see Science and limits below.

Install

Linux on x86_64, Python 3.11 or newer, an NVIDIA GPU and driver. WOOF has no Windows build of the native tools yet.

With Pixi (Python and the Rust/build toolchain managed in one env):

pixi install
pixi run setup
pixi run doctor
# or: pixi run doctor-explain

The default Pixi environment installs this checkout in editable mode with the all-cu12 extra. For a CUDA 13 setup use:

pixi install -e cuda13

On a source checkout, pixi run setup builds the native Rust tools locally. woof fetch-bridges may report that no bundle pins are present; that is normal outside a published wheel install.

python3 -m venv ~/woof-env
~/woof-env/bin/python -m pip install 'recast-woof[all-cu12]'
~/woof-env/bin/woof fetch-tables
~/woof-env/bin/woof fetch-geog --datasets wrf
~/woof-env/bin/woof doctor

For a CUDA 13 setup use all-cu13. The hex model runs on either extra: all-cu12 works on a CUDA 12 or CUDA 13 driver, all-cu13 needs a CUDA 13 driver, and woof hex doctor names the one for your driver. Install one CuPy/CUDA major per environment. The Linux wheel carries the native tools (GRIB and NetCDF decoding, the WPS-equivalent preprocessor, the renderer).

fetch-geog downloads the global geography every forecast builds its terrain, land use and soil from: about 1.3 GB, 17 GB unpacked, once per computer. fetch-tables adds two Thompson microphysics tables (about 314 MiB). Reference tables and map assets arrive with the recast-woof-data package. woof doctor reports each remedy as a command or a # comment explaining the manual step.

For a developer installation from a clone of this repository, use bash install.sh: it installs the checkout's recast-woof-data companion and the package with its extras, builds the native tools (this needs Rust and a C toolchain) and ends with woof doctor. The explicit bash install.sh form also works when an extracted source archive has lost its executable bits; use it if ./install.sh reports Permission denied.

Run a forecast

woof domain --point 35.3,-97.5 --source hrrr --cycle latest --hours 3 \
     --root-dx 3 --point-extent-km 300 --out forecast.toml
woof go forecast.toml --dry-run
woof go forecast.toml

--dry-run reviews the route, sizes it for your card and names anything missing; the second line fetches, prepares, runs and draws. Pictures are drawn as each history frame lands, beside the running forecast.

Coupled hydro is opt-in. woof go forecast.toml --hydro on couples NextGen hydro wherever a hydrofabric covers the domain; a forecast without it has no hydro.

Compare two runs, any product, as run A minus run B:

woof render --diff runA/ runB/ --out diff/

The pair must share the grid and the valid times; a mismatch is refused by name.

Documentation

• docs/public/: the CLI manual, physics, hardware guidance and data sources. The code verification record documents historical WRF v4.6.1 comparisons and their configuration and coverage limits.
• Energy forecasts: woof energy builds 50-100 m forecasts along power lines, substations and renewable sites from OpenStreetMap or your own grid data, with line rating, icing and power products.
• Stage contract for running preprocessing, the simulation and rendering on your own terms.
• Smoke, dust and air quality: wildfire smoke, dust, sea salt, carbon and CAMS air-quality species are an opt-in product, off unless a configuration names chem_sets.
• components/hex/: the hex model's manual, tests and tools.
• woof --help-all lists every command; woof hex --help lists the hex model's commands.

Science and limits

WOOF implements a WRF-ARW-class regional model and WRF-derived physics on the GPU. Comparisons against WRF Fortran, ULP and byte-identity checks, and the matched-run and ensemble consistency comparisons are code verification: evidence that the tested code computes its intended equations. Each result belongs to the build, reference version and configuration it tested. None establishes general equivalence with WRF or skill for every combination of physics, input data and hardware.

Solution verification estimates numerical error in a particular run. Routine numerical-error budgets for weather forecasts remain a gap. Validation compares model results with observations. Meteorology calls that scoring forecast verification; in computational-science V&V terms it is validation. Matching WRF is code verification and never validates a model. WOOF can inherit WRF's published validation record only to the extent the two are statistically indistinguishable for the configuration in question.

Published observation evidence is limited to specific runs and variables, including the ASOS snow/soil comparison, the station/ceilometer fog-suite comparison and an MRMS model-top comparison. The observation scoreboard is in development. These do not establish a broad validation record across regions, seasons or forecast lengths. The evidence page documents historical WRF v4.6.1 code comparisons and their configuration and coverage limits.

The hex model is a GPU port of MPAS-Atmosphere v8.4.2, with MPAS-A's own physics suites as its default and this engine's physics selectable; it is not MPAS and is not endorsed by UCAR. Use official meteorological services for forecasts and warnings.

WOOF continues an engine released before under another name: WOOF 1.0 is that engine's release 2.8.0 and the fixes made since, under the WOOF name, with the hex model inside it. Settings spelled GPUWM_* keep working beside the new WOOF_* spelling.

This fork

Everything above describes Recast WOOF, and is kept as upstream wrote it. This section is the fork's own. Its additions sit on top of the latest upstream release and do not change upstream's defaults: each new option is off until selected, and a configuration that selects none of them runs as upstream's does.

What it adds

• An Apple-silicon backend (MLX/Metal). WOOF_BACKEND=mlx runs the GPU code on a Mac. The CUDA kernels are not rewritten by hand: each .cu unit is translated to Metal when it first loads, and import cupy resolves to a CuPy-compatible layer over MLX. CUDA runs exactly as before. Design and details: docs/dev/mlx-backend.md.
• Further WRF physics, each held word for word to compiled WRF v4.7.1 Fortran where the port says so:
  • SMS-3DTKE (km_opt = 5);
  • use_theta_m = 1 for every ported microphysics and cumulus scheme and for WRF's own input files, plus do_avgflx_em;
  • WRF-Solar: FARMS, the EPS perturbations, MAD-WRF, Deng shallow cumulus and cumulus cloud fed to radiation;
  • WRF v4.7.1's Fitch wind-farm scheme word for word as wind_farm_scheme = "fitch_wrf" beside upstream's fitch, reading WRF's turbine files and layout (or a turbine-library table) from wind_farms, also on hex's unequal-area cells;
  • coupled fire spread and smoke.
• Height-dependent resolution: a partial-depth two-way nest, fine near the ground and coarser aloft, on WRF and (dry dynamics only) on MPAS; and WRF's live vertical nesting (vert_refine_method = 2).
• More data sources: the Met Office global and UKV models, DWD ICON (global and the machine-learning variant) and additional mapped routes.
• Applied tooling: wind-energy siting, hindcasts and power products, storm-track planning with moving nests, and delete-after-commit backpressure and domain selection for upstream's Icechunk store export ([output] icechunk_delete_after_commit, icechunk_domains).
• Continuous integration: GitHub Actions on Linux and macOS, with a self-hosted CUDA job (docs/dev/ci.md).

Install on a Mac

pip install metal-woof        # macOS 14 or newer on Apple silicon, Python 3.11+

The metal-woof wheel carries the macOS builds of the Rust bridges and the MLX stack, and selects the MLX backend on its own. It installs the same woof package and commands as recast-woof; install one or the other, not both. From a checkout, pixi run -e mlx ... uses the same backend.

Limitations of the MLX backend

The Metal backend is younger than the CUDA one and narrower. Before relying on it:

• Apple silicon only, macOS 14 or newer, one GPU. Domain size is bounded by the Mac's unified memory. There is no multi-GPU or peer transport.
• Double precision is emulated. Apple GPUs have no fp64, so double becomes a bit-exact software binary64. It is correct to the last bit and much slower than native fp64; units that lean on it are slow on a Mac.
• The answers are not CUDA's bits everywhere.
  • Metal never contracts a multiply-add, so where a CUDA result carries NVRTC's fused multiply-adds, Metal gives CUDA's --fmad=false words instead. Those families are pinned separately for Metal, each only after a launch-by-launch trace shows the difference is contraction.
  • Single-precision libm calls that are not woof's own glibc ports use Metal's precise:: functions, a recorded vendor-libm difference.
  • NaN payloads can differ.
• The first run compiles. Every kernel unit is translated and compiled on first load. Pipelines are then kept on disk (~/.cache/woof/metal-pipelines), but a cold start can take minutes, and the largest units (CLM lake's step kernel, for one) about twenty.
• The hex (MPAS) model on Metal is opt-in (WOOF_HEX_METAL=1). Only the partial-depth band is measured against MPAS on Metal. The rest of the hex lane runs, but is not yet held to its CUDA words there.
• Virtual Macs are not supported. GitHub's hosted macOS runners expose an "Apple Paravirtual device" that computes wrong answers (NaN) where an M-series GPU matches WRF, so CI runs the Metal parity gates only on real Apple GPUs.
• Some checks do not run on Metal. A few test helpers capture CUDA-only compiler output. They are reported as failing on Metal rather than skipped.

Licence

Apache License 2.0. Third-party code, tables and datasets keep their own terms; see NOTICE and licenses/.