This release is a pre-release and may not be stable for production use.
ACCESS-MOPPy (Model Output Post-processor in Python)
ACCESS-MOPPy is a CMORisation tool designed to post-process ACCESS model output and produce CMIP-compliant datasets.
Key Features
- Python API for integration into notebooks and scripts
- Batch processing system for HPC environments with PBS
- Real-time monitoring with web-based dashboard
- Flexible CMORisation of individual variables
- Dask-enabled for scalable parallel processing
- Cross-platform compatibility (not limited to NCI Gadi)
- CMIP6 and CMIP7 FastTrack support
Installation
ACCESS-MOPPy requires Python >= 3.11.
On NCI Gadi (recommended for ACCESS users)
The conda/analysis3-latest environment maintained by ACCESS-NRI already
includes access_moppy and its dependencies, so no pip install is needed:
module use /g/data/xp65/public/modules
module load conda/analysis3-latest
All command-line tools (moppy-cmorise, moppy-tui, moppy-qc, …) are
available immediately after loading the module. You'll need membership of
the xp65 NCI project for the module itself, plus whichever projects hold
the model output and CV/table data you're processing. Pin a dated release
(e.g. conda/analysis3-26.04) instead of -latest if you need a
reproducible environment for a production run.
From PyPI
pip install access_moppy
From source
The controlled vocabularies under src/access_moppy/vocabularies/ are pulled
in as git submodules. If you install from a local clone, initialise them
first, otherwise the CMOR tables/CVs will be missing and imports will fail
with an error like No module named 'access_moppy.vocabularies.CMIP6_CVs':
git clone --recurse-submodules https://github.com/ACCESS-NRI/ACCESS-MOPPy.git
cd ACCESS-MOPPy
pip install .
If you already have a clone without the submodules populated, run:
git submodule update --init --recursive
pip install .
Quick Start
Interactive Usage (Python API)
import glob
from access_moppy import ACCESS_ESM_CMORiser
# Select input files
files = glob.glob("/path/to/model/output/*mon.nc")
# Create CMORiser instance
cmoriser = ACCESS_ESM_CMORiser(
input_data=files,
compound_name="Amon.pr", # table.variable format
experiment_id="historical",
source_id="ACCESS-ESM1-5",
variant_label="r1i1p1f1",
grid_label="gn",
activity_id="CMIP",
output_path="/path/to/output"
)
# Run CMORisation
cmoriser.run()
cmoriser.write()
Batch Processing (HPC/PBS)
For large-scale processing on HPC systems:
- Create a configuration file (
batch_config.yml):
variables:
- Amon.pr
- Omon.tos
- Amon.ts
experiment_id: piControl
source_id: ACCESS-ESM1-5
variant_label: r1i1p1f1
grid_label: gn
input_folder: "/g/data/project/model/output"
output_folder: "/scratch/project/cmor_output"
file_patterns:
Amon.pr: "output[0-4][0-9][0-9]/atmosphere/netCDF/*mon.nc"
Omon.tos: "output[0-4][0-9][0-9]/ocean/*temp*.nc"
Amon.ts: "output[0-4][0-9][0-9]/atmosphere/netCDF/*mon.nc"
# PBS configuration
queue: normal
cpus_per_node: 16
mem: 32GB
walltime: "02:00:00"
scheduler_options: "#PBS -P your_project"
storage: "gdata/project+scratch/project"
worker_init: |
module load conda
conda activate your_environment
- Submit batch job:
moppy-cmorise batch_config.yml
- Monitor progress at http://localhost:8501
Batch Processing Features
The batch processing system provides:
- Parallel execution: Each variable processed as a separate PBS job
- Real-time monitoring: Web dashboard showing job status and progress
- Automatic tracking: SQLite database maintains job history and status
- Error handling: Failed jobs can be easily identified and resubmitted
- Resource optimization: Configurable CPU, memory, and storage requirements
- Environment management: Automatic setup of conda/module environments
Monitoring Tools
- Streamlit Dashboard: Real-time web interface at http://localhost:8501
- Command line: Use standard PBS commands (
qstat,qdel) - Database: SQLite tracking at
{output_folder}/cmor_tasks.db - Log files: Individual stdout/stderr for each job
File Organization
work_directory/
├── batch_config.yml # Your configuration
├── cmor_job_scripts/ # Generated PBS scripts and logs
│ ├── cmor_Amon_pr.sh # PBS script
│ ├── cmor_Amon_pr.py # Python processing script
│ ├── cmor_Amon_pr.out # Job output
│ └── cmor_Amon_pr.err # Job errors
└── output_folder/
├── cmor_tasks.db # Progress tracking
└── [CMORised files] # Final output
Documentation
- Getting Started:
docs/source/getting_started.rst - Example Configuration:
src/access_moppy/examples/batch_config.yml - API Reference: [Coming soon]
Test Data Override
Integration and end-to-end tests require an external test-data tree set via
the ACCESS_MOPPY_DATA_ROOT environment variable.
- Covered tests: full CMOR integration and end-to-end real-file tests
- No fallback: test-data fixtures in
tests/data/are not used by these tests - Requirement:
ACCESS_MOPPY_DATA_ROOTmust point to a valid dataset root containingoutput*/atmosphere/netCDF,output*/ocean, andoutput*/ice
Example:
export ACCESS_MOPPY_DATA_ROOT=/path/to/CMIP7_Test_data/esm-historical
pixi run -e dev python -m pytest tests/integration/test_full_cmorisation.py
pixi run -e dev python -m pytest tests/integration/test_cmip7_baseline_cmorisation.py
pixi run -e dev python -m pytest tests/e2e/test_end_to_end.py
CMIP7 baseline test note:
tests/integration/test_cmip7_baseline_cmorisation.pyruns one case per CMIP7 baseline variable listed insrc/access_moppy/examples/batch_config_esm1-6_cmip7_baseline.yml- By default, this suite checks end-to-end CMORisation success (run/write/output)
- To additionally enforce WCRP compliance-checker validation for this suite,
set
ACCESS_MOPPY_BASELINE_VALIDATE_WCRP=1
Example with strict WCRP validation enabled:
export ACCESS_MOPPY_DATA_ROOT=/path/to/CMIP7_Test_data/esm-historical
export ACCESS_MOPPY_BASELINE_VALIDATE_WCRP=1
pixi run -e dev python -m pytest tests/integration/test_cmip7_baseline_cmorisation.py --validation-tool=wcrp
Current Status
- Stable project status: ACCESS-MOPPy is suitable for supported CMORisation workflows and ongoing production-oriented use.
- Ocean variables: Ocean variables are supported, including dedicated ocean CMORisers and resource guidance for large 3D variables.
- Variable mapping: Mapping coverage continues to be reviewed and improved for CMIP6/CMIP7 compliance.
Support
- Issues: Submit via GitHub Issues
- Questions: Contact ACCESS-NRI support
- Contributions: Welcome via Pull Requests
License
ACCESS-MOPPy is licensed under the Apache-2.0 License.
1 maintainer
Metadata
1 maintainer
Release files for access-moppy 1.7.18b0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| access_moppy-1.7.18b0.tar.gz | 12.9 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| access_moppy-1.7.18b0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 26.1 MB
Release files / access_moppy-1.7.18b0.tar.gz
| Download URL | access_moppy-1.7.18b0.tar.gz |
|---|---|
| Size | 12.9 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f7b46f8b9ac954eb718373f089cc45d9665efa058d032d301f0740387ca12e9d
|
|
BLAKE2b-256 checksum How to use checksums |
78bc6c8ae20db6d7923d68aded7c6a470d7407d59e829276590c4fddbc659674
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 25, 2026.
Transparency logRelease files / access_moppy-1.7.18b0-py3-none-any.whl
| Download URL | access_moppy-1.7.18b0-py3-none-any.whl |
|---|---|
| Size | 13.3 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
151cfe5c1881c83790b68a3fa3da606655a52e8d4021143137cdb69f1f8fd4b9
|
|
BLAKE2b-256 checksum How to use checksums |
44c4c02584e5352aa5a2af6df70128fd3006324f3f024f3f4efdedefc9c3f691
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 25, 2026.
Transparency log
