PyPartMC 0.3.0

Python interface to PartMC

PyPartMC

PyPartMC is a Python interface to PartMC, a particle-resolved Monte-Carlo code for atmospheric aerosol simulation. PyPartMC is implemented in C++ and it also constitutes a C++ API to the PartMC Fortran internals. The Python API can facilitate using PartMC from other environments - see, e.g., Julia example below.

TL;DR (try in a Jupyter notebook)

! pip install PyPartMC
import PyPartMC

Jupyter notebooks with examples

• Urban plum scenario demo (as in PartMC):
  
• Dry-Wet Particle Size Equilibration with PartMC and PySDM:
  

Features

• works on Linux, macOS and Windows (compatibility assured with CI builds)
• hassle-free installation using pip (prior PartMC installation not needed)
• works out of the box on mybinder.org, Google Colab and alike
• ships with a set of examples maintained in a form of Jupyter notebooks
• Pythonic API (but retaining PartMC jargon) incl. Python GC deallocation of Fortran objects
• specification of parameters using native Python datatypes (lists, dicts) in place of PartMC spec files
• code snippets in README depicting how to use PyPartMC from Julia (also executed on CI)
• auto-generated API docs on the web
• support for [de]serialization of selected wrapped structures using JSON
• based on unmodified PartMC code
• does not use or require shell or any pre-installed libraries
• aiming at 100% unit test coverage

Usage examples

example object instantiation in Python

import PyPartMC as ppmc
print(ppmc.__version__)
gas_data = ppmc.GasData(("H2SO4", "HNO3", "HCl", "NH3", "NO", "NO2"))

example object instantiation in Julia

using Pkg
Pkg.add("PyCall")

using PyCall
ppmc = pyimport("PyPartMC")
print(ppmc.__version__)
gas_data = ppmc.GasData(("H2SO4", "HNO3", "HCl", "NH3", "NO", "NO2"))

usage in other projects

PyPartMC is used within the test workflow of the PySDM project.

Implementation outline

• PyPartMC is written in C++, Fortran and uses pybind11 and CMake.
• JSON support is handled with nlohmann::json and pybind11_json
• PartMC and selected parts of SUNDIALS are statically linked (and compiled in during pip install or python -m build)
• C (SUNDIALS, netCDF), C++ (pybind11, ...) and Fortran (PartMC, CAMP, netCDF-fortran) dependencies are linked through git submodules
• MOSAIC dependency is optionally linked through setting the environmental variable MOSAIC_HOME
• a drop-in replacement of the PartMC spec file routines is used for i/o from/to JSON

Implementation architecture

flowchart TD
    subgraph J ["Julia"]
        julia_user_code["Julia user code"] --> PyCall.jl
    end
    subgraph P ["Python"]
        PyCall.jl --> PyPartMC
        python_user_code -.-> NumPy
        python_user_code["Python user code"] ---> PyPartMC["pubind11-generated PyPartMC module"]
    end
    subgraph Cpp ["C++"]
        cpp_user_code["C++ user code"] ----> ppmc_cpp
        PyPartMC --> ppmc_cpp["PyPartMC-C++"]
        ppmc_cpp --> pybind11_json
        pybind11_json ---> nlohmann::JSON
        fake_spec_file_cpp --> nlohmann::JSON
    end
    subgraph C ["C"]
        fake_spec_file_c --> fake_spec_file_cpp["SpecFile-C++"]
        ppmc_cpp --> ppmc_c["PyPartMC-C"]
        netCDF-C
        SUNDIALS
        camp_c["CAMP C code"]
    end
    subgraph Fortran ["Fortran"]
        PartMC -....-> MOSAIC
        ppmc_c --> ppmc_f["PyPartMC-F"]
        ppmc_f ---> PartMC
        PartMC --> netCDF-F
        netCDF-F --> netCDF-C
        PartMC --> SUNDIALS
        PartMC ---> camp_f
        camp_f["CAMP"] --> camp_c
        PartMC ----> fake_spec_file_f[SpecFile-F]
        fake_spec_file_f --> fake_spec_file_c["SpecFile-C"]
    end

    style PartMC fill:#7ae7ff,stroke-width:2px,color:#2B2B2B

Troubleshooting

Common installation issues

error: [Errno 2] No such file or directory: 'cmake'

Try rerunning after installing CMake, e.g., using apt-get install cmake (Ubuntu/Debian), brew install cmake (homebrew on macOS) or using MSYS2 on Windows.

No CMAKE_Fortran_COMPILER could be found.

Try installing a Fortran compiler (e.g., brew reinstall gcc with Homebrew on macOS or using MSYS2 on Windows).

Could not find NC_M4 using the following names: m4, m4.exe

Try installing m4 (e.g., using MSYS2 on Windows).

Notes for developers

How to debug

git clone --recursive git+https://github.com/open-atmos/PyPartMC.git
cd PyPartMC
DEBUG=1 VERBOSE=1 pip --verbose install -e .
gdb python 
(gdb) run -m pytest -s -vv -We -p no:unraisableexception tests

Pre-commit hooks

PyPartMC codebase benefits from Pylint, Black and isort code analysis (which are all part of the CI workflows where we also use pre-commit hooks. The pre-commit hooks can be run locally, and then the resultant changes need to be staged before committing. To set up the hooks locally, install pre-commit via pip install pre-commit and set up the git hooks via pre-commit install (this needs to be done every time you clone the project). To run all pre-commit hooks, run pre-commit run --all-files. The .pre-commit-config.yaml file can be modified in case new hooks are to be added or existing ones need to be altered.

Credits

PyPartMC:

authors: PyPartMC developers
funding: US Department of Energy Atmospheric System Research programme
copyright: University of Illinois at Urbana-Champaign
licence: GPL v3

PartMC:

authors: Nicole Riemer, Matthew West, Jeff Curtis et al.
licence: GPL v2 or later