PySPD

New Zealand electricity dispatch and pricing in Python.

PySPD is a class-based Pyomo implementation of New Zealand's Scheduling, Pricing, and Dispatch model. Use it to replay vSPD inputs, inspect dispatch and prices, and build reproducible market studies with explicit input, solver, and report provenance.

Get started · Documentation · Case studies · Validation status

[!IMPORTANT] PySPD is an engineering candidate under staged validation. Retained comparisons support specific dates, formulations, and report surfaces; complete historical parity is not established. The qualified execution environment is Python 3.13 on macOS ARM64.

What it does

• Reads v5-style pricing GDX and legacy v3 final-pricing inputs through explicit schema adapters.
• Assembles energy, AC/HVDC network, reserve, and version-specific algebra from typed model components.
• Solves dispatch with SCIP, then fixes discrete/SOS state and prices the resulting continuous model with HiGHS.
• Checks physical residuals, objectives, prices, and reports independently.
• Writes twelve deterministic CSV tables and a SHA-256-bound manifest.json.
• Runs independent pricing cases in parallel and supports audited counterfactuals through a lower-level Python API.

GDX → schema validation → preprocessing → Pyomo model
    → SCIP dispatch → fixed-discrete HiGHS pricing
    → independent validation → prices and report bundle

Installation

Install in a Python 3.13 environment:

pip install pyspd

SCIP (pyscipopt) and HiGHS (highspy) install automatically with PySPD. For GDX input support, use pip install "pyspd[gdx]" and configure your local GAMS runtime. The clp, cbc, and probity extras enable additional APIs and solver profiles. See the installation guide.

Quick start

The qualified execution environment is macOS ARM64 with Python 3.13. Install uv and a local GAMS runtime, then obtain a vSPD-compatible GDX input. GAMS provides GDX access; the normal solve uses SCIP and HiGHS and does not require CPLEX.

git clone https://github.com/waikato-ahuora-smart-energy-systems/pySPD.git
cd pySPD
uv sync --frozen --group gdx
uv run pyspd formulations --json

Create run.json, replacing the paths and SHA-256 with your own values:

{
  "formulation_id": "vspd-v5.0.6-reserve",
  "input_path": "/absolute/path/Pricing_20230927.gdx",
  "output_directory": "/absolute/path/results/20230927",
  "source_sha256": "replace-with-the-64-character-input-sha256",
  "gams_system_directory": "/Library/Frameworks/GAMS.framework/Resources",
  "case_ids": [],
  "worker_count": 1
}

The default schema is vspd-v5.0.6 and the default solver profile is scip-mip-fixed-highs-rmip. An empty case_ids list selects the complete supported day; selecting one known case is a faster first check.

uv run pyspd run --config run.json
# For a full day on a host with sufficient memory:
uv run pyspd run --config run.json --workers 10

The first-run guide includes input hashing, case inventory, a configuration generator, and output verification.

Choose a workflow

The production formulations are vspd-v5.0.6-reserve and spd-v16.0-reserve. The multi-period battery study is a separate analytic research profile. See current limitations for input, platform, scenario, and historical-report boundaries.

External evidence

Large historical inputs and detailed CPLEX/solver observations are kept as immutable, hash-bound release assets:

uv run pyspd evidence list
uv run pyspd evidence fetch cplex-reference-v1 --destination .

Ordinary tests do not download archives. Missing external evidence produces an explicit skip, which is not a passing oracle result. The external evidence guide explains archive selection, authentication, caching, and verification.

Build the documentation

The Read the Docs site uses Sphinx with the Read the Docs theme and the docs dependency group:

uv sync --frozen --group docs
uv run --no-sync sphinx-build -W --keep-going -b html docs site
uv run --no-sync python -m http.server 8765 --bind 127.0.0.1 --directory site

See maintaining the documentation for site structure, build configuration, and evidence-file handling.

Development

uv sync --frozen --group docs
uv run --no-sync ruff check .
uv run --no-sync mypy src tools
uv run --no-sync pytest -q
uv run --no-sync python -m tools.probity_audit
uv run --no-sync sphinx-build -W --keep-going -b html docs site
git diff --check

The default dependency groups include development tools, HiGHS, CLP, and CBC. Add --group gdx when the work needs GDX access. Model-affecting changes follow Probity TDD: preserve the failing test before implementation and retain the red/green evidence with its requirement, environment, and commits.

Internal engineering records are maintained in private/, separately from the user documentation published to Read the Docs.

Licence

PySPD is licensed under the Apache License 2.0. Third-party dependencies, external input data, and solver runtimes retain their own terms.