Skip to main content
Yanked

This release has been yanked by its maintainers, and will be ignored by installers, except when explicitly specified.
Consider using release 0.1.1 instead.
Reason given by maintainers: description 更新

xmetai-inference

English | 简体中文

Inference framework for autoregressive weather forecasting models. It reads ERA5 / CRA-40 reanalysis from Zarr stores, feeds each model the channel set and history window its contract declares, runs the rollout, and writes the forecast as NetCDF.

  • Runs ONNX and torch.export (.pt2) graphs behind one interface
  • Supports deterministic and 51-member ensemble models on a 0.25° global grid
  • Ships a run recipe per model, selected with --config <name>
  • Multi-GPU by sharding initialization times × members across cards

Release version: 0.1.0.

Exported ONNX models require GPU inference support. Use xmetai-onnx-plugins to provide the custom GPU operators required by the corresponding models.

Project Purpose
xmetai-core Core framework for meteorological AI model training and inference
xmetai-skills Skills for model development
xmetai-datakit Data processing and evaluation
xmetai-onnx-plugins Custom GPU operators for exported ONNX models

Table of Contents

Quick Start

Requirements

  • Linux (or WSL2 / Git Bash on Windows)
  • An NVIDIA GPU with CUDA
  • Model weights and the input Zarr stores (both excluded from git)

Install

From PyPI:

pip install xmetai-inference
pip install "xmetai-inference[netcdf,zarr,onnx]"   # every model except FuXi-2.1 (ONNX)
pip install "xmetai-inference[netcdf,zarr]"        # NetCDF I/O only: FuXi-2.1 / evaluation

From a source checkout, for development:

pip install -e .
pip install -e ".[netcdf,zarr,onnx]"
pip install -e ".[netcdf,zarr]"

torch comes in with the base install. The netcdf extra brings netCDF4 / h5netcdf (reading mean.nc/std.nc, writing output), zarr powers store reads in xmetai-core, and onnx pulls in onnxruntime.

PyPI's onnxruntime is the CPU build. For GPU inference install onnxruntime-gpu instead (the onnx extra and onnxruntime-gpu conflict; use one or the other). The loader logs a warning when it has to fall back to the CPU provider.

XMetAI ONNX Runtime plugin (custom operators). Recipes that set ops_library (the Tongyun recipes and fengshun2.0) load custom operators through the XMetAI ONNX Runtime plugin. Download the build matching your ONNX Runtime / CUDA version from https://github.com/XMetAI-Lab/xmetai-onnx-plugins/releases/ and point the recipe's ops_library at the downloaded .so (for example xmetai_onnx_plugins.ort1_24.cuda12.so). Without the plugin the graph fails to load with an unknown-operator error.

Run

Every supported model has a ready-made recipe (configs/<name>.py):

xmetai-infer --config weather_era_0p25_fuxi_ens
xmetai-infer --config weather_era_0p25_fuxi21
xmetai-infer --config weather_era_0p25_tongyun
xmetai-infer --config weather_era_0p25_fengqing
xmetai-infer --config weather_era_0p25_pangu
xmetai-infer --config weather_era_0p25_tongyun_ens
xmetai-infer --config weather_cra_0p25_tongyun_ens
xmetai-infer --config s2s_cra_0p25_fengshun20_ens
xmetai-infer --config weather_nc_fuxiens         # example: official FuXi-ENS input.nc
xmetai-infer --config weather_nc_fuxi21          # example: official FuXi-2.1 input.nc

Each recipe declares the weight path, dataset and preprocessing chain, initialization times, step count, GPU count, and output directory. The shipped recipes use /path/to/... placeholders — point those at your own weights and stores (or set the matching environment variables) before the first run. From a source checkout without installing:

python -m xmetai_inference --config weather_era_0p25_fuxi21

Output goes to {out}/{YYYYMMDD|YYYYMMDDHH}/[member_XXX/]{step:03d}.nc, one NetCDF per step.

xmetai-infer --worker ... is the internal worker subprocess entry point used for every run (single- or multi-GPU) and should not be called manually.

Examples

Two self-contained example recipes infer straight from one given .nc (dataset.type="netcdf_data") — no Zarr store. Both sample files ship with the corresponding official upstream release; this repository does not host them:

Recipe Model (registry) Files members × steps
weather_nc_fuxiens FuXi-ENS (fuxi_ens_onnx) official data/input.nc + model/fuxi_ens.onnx (and its external weights model/fuxi_ens) 2 × 2
weather_nc_fuxi21 FuXi-2.1 (fuxi21_pt2) official input.nc + fuxi-2.1.pt2 + mean.nc/std.nc 1 × 2

Where the sample .nc files come from:

  • FuXi-ENS — data/input.nc (2 × 78 × 721 × 1440, initialization 2018-09-14 00Z) ships in FuXi-ENS-main.zip from the official model record Zenodo 15124541 (the release accompanying the Science Advances paper, DOI 10.1126/sciadv.adu2854), together with model/fuxi_ens.onnx and its external weight file model/fuxi_ens. The bundled README additionally points to a restricted Google Drive folder (contact Prof. Li Hao, lihao_lh@fudan.edu.cn).
  • FuXi-2.1 — input.nc (2 × 85 × 721 × 1440, initialization 2024-09-29 00Z, already z-score normalized) ships in the Hugging Face repository tpys/fuxi-2.1 together with fuxi-2.1.pt2, mean.nc and std.nc; the model is part of the FuXi Single collection.

Neither recipe preprocesses the nc in the dataset layer. FuXi-ENS normalizes inside the ONNX graph (physical in, physical out); FuXi-2.1's input.nc is already z-score normalized and its output is denormalized with the mean.nc/std.nc next to the .pt2. The nc alone supplies the history window; the init time is its last time frame, so cfg.times is empty. The FuXi-ENS example runs 2 members × 2 steps as a smoke test — use --members 51 --steps 60 (or edit the recipe) for a full run; the graph samples noise internally.

Point SAMPLE_ROOT in each recipe at the directory holding the upstream release files:

  • FuXi-ENS — input.nc, fuxi_ens_onnx/fuxi_ens.onnx, and the external weight file fuxi_ens it references (≈9.3 GiB, extensionless, must sit next to the .onnx).
  • FuXi-2.1 — input.nc, mean.nc, std.nc, and fuxi-2.1.pt2 (3.7 GiB).

Then one command each:

python -m xmetai_inference --config weather_nc_fuxiens --out <output_dir>
python -m xmetai_inference --config weather_nc_fuxi21  --out <output_dir>

Output lands at {out}/{YYYYMMDD|YYYYMMDDHH}/[member_XXX/]{step:03d}.nc (no member_XXX layer when members=1).

An input nc only works if it matches the graph: physical quantities (or normalized values for FuXi-2.1) in input_channels order, the model's grid orientation, float32, (time=history, channel=N, 721, 1440), and sst cold-spots filled where the graph expects it.

Override Parameters

xmetai-infer --config weather_era_0p25_fuxi_ens \
  --times "2025010600..2025011200:24" \
  --steps 60 --members 51 --gpus 4 --cuda-devices 0,1,2,3 \
  --vars z500,u200,v200,msl,tp \
  --out <output_dir>
  • --times follows the parse_times syntax in xmetai_inference/configs/base.py: a single 2025010600, a comma-separated list, a closed interval A..B (24 h step by default), or :N to override the step.
  • --config takes a built-in recipe name or any external .py recipe; the positional argument is the same thing for a file path. See configs/ for examples.
  • --model-path swaps the weight file without touching the recipe.

Supported Models

Model Recipe (--config) Backend Type Members Channels Grid Step / history Normalization
FuXi-Ens weather_era_0p25_fuxi_ens onnx Ensemble 51 78 GRID_025 6h / 2 frames Baked into graph; physical units in/out
FuXi-2.1 weather_era_0p25_fuxi21 pt2 Deterministic 1 85 GRID_025 6h / 2 frames Data-side normalize, output denormalize (mean.nc/std.nc next to the weights)
Tongyun (det) weather_era_0p25_tongyun onnx Deterministic 1 78 GRID_025 6h / 2 frames Baked into graph; custom ops (GatedDeltaNet2Fn) require ops_library
FengQing V1.5Beta weather_era_0p25_fengqing onnx Deterministic 1 70 GRID_025 6h / 2 frames normalize_fengqing / denormalize_fengqing (per-pixel stats; graph emits a residual)
Pangu-Weather weather_era_0p25_pangu onnx Deterministic 1 69 GRID_025 6h / 1 frame (+24h jump) Baked into graph; physical units in/out
Tongyun-Ens weather_era_0p25_tongyun_ens onnx Ensemble 51 78 GRID_025 6h / 2 frames Baked into graph; physical units in/out; custom ops require ops_library
Tongyun-Ens (CRA) weather_cra_0p25_tongyun_ens onnx Ensemble 51 71 GRID_025_S 6h / 2 frames Store-normalized → data-side denormalize; graph normalizes at its head, outputs physical units
fengshun2.0 s2s_cra_0p25_fengshun20_ens onnx Ensemble 51 87 GRID_025 24h / 2 frames Store-normalized → data-side denormalize (tp via expm1); graph normalizes at its head

The ERA5 models share the 0.25° global regular grid (721×1440, latitude north→south, longitude 0–360°). All use a 2-frame history window except Pangu, which uses a single frame (history_steps=1) and, every 4 steps (lead 24 h), switches to a 24 h model that jumps directly from the anchor (official "Plan B").

Three exceptions worth knowing:

  • Tongyun-Ens (CRA) uses GRID_025_S (latitude south→north, the store's native orientation) and its preprocessing must not configure geometry — that flips the field in reverse.
  • fengshun2.0 runs on a daily step (hour_interval=24), so steps=60 is a 60-day forecast. Its store is CRA north→south, so it also skips geometry. The recipe asks for 51 members even though the model class declares members=11 as the graph's native member axis.
  • Pangu needs a second weight file for the 24 h jump (PANGU_ONNX_24); it defaults to pangu_weather_24.onnx next to the 6 h model.

Channel Sets and Order

Every model's input_channels (also output_channels) is the flattened list below; the order is part of the contract. A wrong order does not raise — it silently forecasts the wrong field.

upper = [f"{var}{level}" for var in PL_VARS for level in LEVELS]      # 65 channels

FuXi-Ens  (78): PL z,t,u,v,q × [50,100,150,200,250,300,400,500,600,700,850,925,1000]
                + [t2m, d2m, sst, u10m, v10m, u100m, v100m, msl, ssr, ssrd, fdir, ttr, tp]
FuXi-2.1  (85): PL z,t,u,v,q × same levels
                + [msl, t2m, d2m, sst, ws10m, ws100m, u10m, v10m, u100m, v100m,
                   lcc, mcc, hcc, tcc, ssr, ssrd, fdir, ttr, tcw, tp]
fengshun  (87): PL z,t,u,v,q × same levels
                + [msl, t2m, t2m_min, t2m_max, d2m, sst, ws10m, ws100m, u10m, v10m,
                   u100m, v100m, lcc, mcc, hcc, tcc, ssr, ssrd, fdir, ttr, tcw, tp]
FengQing  (70): PL z,q,t,u,v × [1000,925,850,700,600,500,400,300,250,200,150,100,50]
                + [tp, msl, u10m, v10m, t2m]
Pangu     (69): PL z,q,t,u,v × same descending-pressure levels
                + [msl, u10m, v10m, t2m]
Tongyun   (71): PL z,t,u,v,q × [50 … 1000] (ascending pressure)
                + [t2m, q2m, u10m, v10m, msl, tp]

Channel names are matched against store names by _raw_channel_name: exact match first, then z50 → z_50. A store may be a superset of the model channels — every requested channel must exist, or construction fails loudly. Two processors read names differently: denormalize aligns store stats by underscore-insensitive, lower-cased name and tolerates a superset; normalize requires stats that match input_channels in count and order exactly.

Model naming. Config names follow <domain>_<source>_<resolution>_<model>[_ens]: domain = weather (15-day, 6-hourly) or s2s (subseasonal, daily steps); source = era (ERA5) or cra (CRA-40); resolution = 0p25 (0.25°) or 1p5. The trailing _ens marks an ensemble output when the model name doesn't already contain ens; deterministic configs carry no suffix. Examples: weather_era_0p25_fuxi_ens, weather_era_0p25_tongyun_ens, weather_cra_0p25_tongyun_ens, s2s_cra_0p25_fengshun20_ens. Each maps to the eponymous recipe under configs/. Model-class registry names (the model_class= in recipes, e.g. fuxi_ens_onnx, fuxi21_pt2, iwc_tongyun_gdn2_onnx (the Tongyun det graph), fengqing_pre_onnx, pangu_onnx, tongyun_era_ens_onnx, tongyun_cra_ens_onnx, fengshun20_onnx) map to MODEL_REGISTRY in utils/model_factory.py.

Supported Datasets

Two dataset types: processed_multi_zarr (the production path, backed by xmetai-core's MultiZarrDataset) and netcdf_data (one given .nc per initialization, for demos and business-supplied inputs).

processed_multi_zarr

The stores are Zarr products built to the xmetai-datakit data contract. The contract defines the array structure, channel order, units, normalization, masks, and metadata a product must satisfy: see config/dataset_specs.

dataset = {
    "type": "processed_multi_zarr",
    "paths": ["<store_root>/store_a.zarr", "<store_root>/store_b.zarr"],
    "processors": [ ... ],          # executed in the DataLoader worker, in order
}
  • paths may list several stores. Each is opened and sliced to the requested time range; their time coordinates are aligned to the intersection and to freq, then the arrays are concatenated along channel. The first store listed wins on duplicate channel names.
  • One sample is hist_frames consecutive frames ending at the init time, read as (hist_frames, C, lat, lon) float32 (e.g. 2 × 78 × 721 × 1440 for FuXi-Ens). The init time is the last history frame; the store must contain at least one frame after it. A requested init time without enough history raises.
  • The framework derives hist_frames, fcst_frames=1, freq, interval=1, inference_only=True, years, and in_names=[] from the model contract and the requested init times. Do not repeat these keys in dataset (duplicate keyword arguments); any other key is forwarded to MultiZarrDataset as-is (e.g. bbox, lat_flip).

netcdf_data

Infer straight from given .nc files, one file per initialization:

dataset = {
    "type": "netcdf_data",
    "paths": ["/abs/input.nc"],
}

Each file holds the full model input window [time, channel, lat, lon]; all history frames live inside the nc itself (the engine never reads back into a store). The init time is the last time frame's value, so cfg.times is not consulted for this type. Values pass through untouched — no processor chain runs. Only the channel axis is verified and rewritten to input_channels by name (underscore-insensitive, case-insensitive); a missing channel or a count mismatch fails at build time. Samples are built eagerly once at dataset build time, so this type is single-process-only.

Project Structure

.
├── pyproject.toml           # PyPI build metadata and CLI entry
├── xmetai_inference/        # The installable Python package (package name: xmetai-inference)
│   ├── cli.py               # Config loading, multi-GPU scheduling, the autoregressive main loop, async disk-write
│   ├── data/                # Dataset implementations, grouped into subpackages by category (weather/)
│   ├── utils/               # dataloader_factory (config -> DataLoader, dispatched by dataset.type),
│   │                        #   runner (Rollout / Trajectory autoregressive orchestration),
│   │                        #   model_factory (MODEL_REGISTRY / get_model_class / create_model),
│   │                        #   model-side pipeline (recurrent/output), data-side tensor processors, logging
│   ├── backends/            # BaseInferModel + onnx / pt2 execution engines
│   └── configs/             # Run recipes (`--config <name>`) + declared model contracts;
│                            #   base.py = shared GRID/channels + InferConfig/load_config
├── tests/                   # Unit + integration tests (integration runs only with `-m realdata`)
└── model_artifacts/         # Local model artifacts; absent in a fresh clone and excluded from Git

How It Works

One forecast, from store to disk:

zarr store  →  xmetai-core MultiZarrDataset          (read history frames, select variables by model channel)
            →  dataset.processors                    (in DataLoader worker, torch; chain depends on store layout)
                 unit_convert → geometry → normalize → fill_missing     ← raw store (ERA5)
                 denormalize (stats_path=store, ± expm1) → …            ← normalized store (CRA / c87 / v3)
            →  prepare_dataset_batch                 ([B,T,C,H,W] tensor)
            →  Rollout (autoregressive)              (state = result feedback)
                 recurrent processors (zero_channels)
            →  output processors (denormalize)       (workspace → physical units)
            →  to_dataset                            (xarray, non-negative fallback for tp)
            →  _AsyncWriter                          (single background thread writes NetCDF)
  • Contract-driven models: channels, grid, history window, member semantics, and state representation are declared by the model class, so backends stay interchangeable.
  • Pluggable backends: onnx and pt2 execution engines are decoupled from model semantics.
  • One recipe per run: the config declares dataset (incl. processors), dataloader, and model_processing (the recurrent and output stages).
  • Unified autoregression: utils/runner.py's Rollout/Trajectory chain single-step forward into a multi-trajectory loop with state = result feedback.
  • Async disk-write: one background thread writes NetCDF; on failure it falls back to per-step raw_step_XXX.npy.
  • Multi-GPU data parallelism: work is sharded by "initialization time × member"; each worker process is pinned to one card via CUDA_VISIBLE_DEVICES.

Input preprocessing goes through dataset.processors; model_processing.input must be empty (misconfiguring it errors out). Normalization already fused into the model file is not re-executed by the framework.

License and contact

Licensed under GPL-3.0-or-later; see LICENSE. Model weights, data, and third-party assets may have separate terms.

XMetAI Team: douzsh@gmail.com.

Metadata

Release files for xmetai-inference 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xmetai-inference 0.1.0
File Size Uploaded Provenance Actions
xmetai_inference-0.1.0.tar.gz 85.9 kB With provenance Details for xmetai_inference-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for xmetai-inference 0.1.0
File Interpreter ABI Platform Actions
xmetai_inference-0.1.0-py3-none-any.whl Python 3 none any Details for xmetai_inference-0.1.0-py3-none-any.whl

Total release size: 175.7 kB

Release files / xmetai_inference-0.1.0.tar.gz

Download URL xmetai_inference-0.1.0.tar.gz
Size 85.9 kB
Tags Source
SHA-256 checksum
How to use checksums
f0986281f8ed42f5f76f28890813b7c6a9dce0a8a8426a0d86e80b9ecbbba2ee
BLAKE2b-256 checksum
How to use checksums
ae464ca47b356ddb5be94827cc2f3aa4a8fe2bbf945f36b39b9670bea9aea1d5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Release files / xmetai_inference-0.1.0-py3-none-any.whl

Download URL xmetai_inference-0.1.0-py3-none-any.whl
Size 89.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3c7ad5c2e3d3c1c8774ddf0d808d05eb3bcd832658f15830719f679ed3b1e48e
BLAKE2b-256 checksum
How to use checksums
e8660f2bcceb8d930f035e5139a80015ba76c78940f97f8a3f91e50b0c54fdf9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Security for xmetai-inference 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for xmetai_inference-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page