MinxiongHydroCast

Official-source hydrometeorological observations and reproducible rainfall-nowcasting data and model development for Minxiong, Taiwan.

Status: Operational Prototype / Data and Model Development. The observation service is usable on a localhost-only deployment. Forecast publication and automated risk notifications remain disabled until the model, label, and shadow-deployment gates pass.

Real localhost operator view captured on 2026-07-29. It uses live official observations; values change over time. The blocked shadow gate is intentional.

What problem does this solve?

Official rainfall, radar, warning, and flood-sensor feeds are useful but have different schemas, cadences, failure modes, and retention windows. A downloader alone cannot answer whether a snapshot is fresh, internally consistent, reproducible, or safe to expose.

MinxiongHydroCast turns those feeds into a fail-closed observation, data, and model system:

• strict Pydantic contracts, freshness checks, and cross-page WRA sensor joins;
• bounded retries and explicitly degraded fallbacks without hiding schema drift;
• immutable snapshots with SHA-256, source authority, dataset ID, fetch time, and adapter version;
• CLI, read-only API, health/readiness endpoints, Prometheus metrics, backup, and operator view;
• reproducible CWA radar event datasets with human-reviewed evidence and fixed event splits;
• Persistence and Tiny U-Net evaluation behind promotion gates that can block publication.

That operational and scientific boundary is the main difference from a general weather-data download script.

Architecture

flowchart LR
    subgraph Official["Official sources"]
        CWA["CWA Open Data<br/>gauges · radar · QPE"]
        WRA["WRA Open Data<br/>warnings · flood sensors"]
    end

    CWA --> INGEST["Adapters<br/>bounded retries"]
    WRA --> INGEST
    FALLBACK["WRA page parser<br/>degraded diagnostics"] -. transport-only fallback .-> INGEST
    INGEST --> CONTRACTS["Strict contracts<br/>schema · freshness · joins"]
    CONTRACTS --> SNAPSHOTS["Immutable snapshots<br/>provenance · SHA-256"]
    SNAPSHOTS --> SERVICE["Read-only service<br/>API · health · readiness · metrics · UI"]
    CONTRACTS --> EVIDENCE["Radar event evidence<br/>human review"]
    EVIDENCE --> DATASET["Reproducible event splits<br/>tensor archives"]
    DATASET --> MODELS["Persistence<br/>weighted Tiny U-Net"]
    MODELS --> GATES{"Model + label +<br/>shadow gates"}
    GATES -- pass --> FORECAST["Experimental forecast API"]
    GATES -- blocked --> CLOSED["Forecast unavailable"]

Schema drift, invalid units or timestamps, broken measurement/catalog joins, and unexpected empty observation sets fail the attempt. The optional scraper fallback is limited to transport, authentication, timeout, HTTP, or rate-limit failures and never satisfies readiness. See the architecture and data contracts.

Official data flow

API keys, official raw files, event evidence, model weights, live snapshots, CCTV, and host configuration are not committed. The source register records authority, acceptance, and redistribution questions.

Current status

Public-safe verification on 2026-07-29:

These are dated observations, not an availability promise. The current public-safe rollout record is in deployment status.

Baseline results

The formal experiment uses six radar input frames to predict six target frames at 10-minute cadence. Metrics below use independent validation/test events; lower RMSE and higher CSI are better.

The weighted Tiny U-Net lowers aggregate RMSE on all three events, but CSI regresses on one local test event and some 10-to-60-minute lead-time gates regress. Therefore forecast_publication_ready=false; Persistence remains the required benchmark. See the full baseline results, model card, and data assets.

Quick Start

Python 3.11 and 3.13 are tested in CI.

The fastest path is a credential-free, synthetic demo:

docker compose up --build

Open http://127.0.0.1:8080/. The dashboard shows demo rain gauges and flood sensors, /healthz, intentionally blocked /readyz, Prometheus /metrics, and the forecast publication gate. No API key or live official request is used.

If port 8080 is already in use, choose another host port:

MHC_DEMO_PORT=18080 docker compose up --build

A 60-second capture of the credential-free synthetic stack. Every source is classified as demo_fixture; readiness and forecast publication stay blocked.

For a local Python installation:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"

Create a deterministic demo snapshot without contacting live sources, then open the operator view:

mhc collect --region minxiong --mode demo --once
mhc serve --host 127.0.0.1 --port 8080

In another terminal:

curl --fail http://127.0.0.1:8080/healthz

Open http://127.0.0.1:8080/. Demo data intentionally does not pass readiness. Live collection requires CWA/WRA credentials in an ignored .env; follow operational use rather than placing secrets on the command line.

Useful entry points:

mhc --help
mhc collect --help
mhc serve --help
mhc dataset build --help
mhc data relocate-root --help
mhc event queue --help
mhc model evaluate --help
mhc model evaluate-optical-flow --help
mhc model optical-flow-report --help
mhc operations backup --help

The base wheel installs only Pydantic, Requests, and NumPy. Install capability extras only when needed:

pip install "minxiong-hydrocast[scraper]"
pip install "minxiong-hydrocast[model]"
pip install "minxiong-hydrocast[report]"

Example output

A live /readyz response can be reduced to the service contract with:

curl --silent http://127.0.0.1:8080/readyz |
  jq '{state, ready, latest_snapshot: {mode: .latest_snapshot.mode},
       latest_attempt: {status: .latest_attempt.status}}'

{
  "state": "healthy",
  "ready": true,
  "latest_snapshot": {"mode": "live"},
  "latest_attempt": {"status": "ok"}
}

The service also exposes /healthz, /metrics, /api/v1/status, official observations, region features, locations, shadow readiness, and a fail-closed experimental forecast endpoint.

Evaluation and tests

python -m compileall -q src tests scripts
python -m ruff check .
python -m pytest -q

CI runs the same quality gates on Python 3.11 and 3.13. CodeQL, Dependabot, secret scanning, and protected main rules provide repository-level controls. A separate clean-wheel job builds both distributions, installs the wheel, verifies that mhc is the only executable, and exercises the synthetic API/readiness/metrics/forecast-gate flow. Scheduled live-contract checks detect upstream CWA/WRA changes without printing credentials.

Limitations

• This is not an official warning system, public forecast service, or emergency decision tool.
• Five radar events do not cover enough typhoon, frontal, Mei-yu, and convective regimes.
• Radar reflectivity is not surface rainfall or flood depth; QPE/gauge validation is incomplete.
• Reviewed local flood labels have not reached the 10-positive / 20-negative minimum.
• The rolling shadow gate still lacks a confirmed heavy-rain period and has a gap above 30 minutes.
• Official data and trained-weight redistribution rights require separate review.
• The supplied deployment profile is localhost-only; public ingress requires authentication, TLS, ownership, incident response, and completed gates.

Data, model, and code license

Repository code is released under the MIT License. That license does not relicense CWA or WRA data, third-party documents, event evidence, or trained weights. This repository ships schemas and synthetic samples, not an official dataset or model checkpoint. Review the data source register and each authority's terms before redistribution or commercial use.

Roadmap and releases

See CHANGELOG.md, the v0.1.5 release notes, v0.1.4 release notes, v0.1.3 release notes, the v0.1.2 release notes, and the long-term roadmap. Current work belongs in tasks; generated deployment numbers do not belong in the README.

Documentation

• Start here: documentation index, project scope, architecture
• Operate: operational use, single-host runbook, incident response, rollback
• Contracts: data contracts, source register, spatial alignment, region profiles, adapter development
• Data and models: dataset build, storage layout, data-root relocation, event evidence and review, baseline results, model card
• Governance: decision authority, security policy, contributing, roadmap, tasks