PyPartMC 1.3.2

Python interface to PartMC

PyPartMC

PyPartMC is a Python interface to PartMC, a particle-resolved Monte-Carlo code for atmospheric aerosol simulation. PyPartMC is implemented in C++ and it also constitutes a C++ API to the PartMC Fortran internals. The Python API can facilitate using PartMC from other environments - see, e.g., Julia and Matlab examples below.

For an outline of the project, rationale, architecture, and features, refer to: D'Aquino et al., 2024 (SoftwareX) (please cite if PyPartMC is used in your research). For a list of talks and other relevant resources, please see project Wiki. If interested in contributing to PyPartMC, please have a look a the notes for developers.

TL;DR (try in a Jupyter notebook)

! pip install PyPartMC
import PyPartMC

Jupyter notebooks with examples

Note: clicking the badges below redirects to cloud-computing platforms. The mybinder.org links allow anonymous execution, Google Colab requires logging in with a Google account, ARM JupyerHub requires logging in with an ARM account (and directing Jupyter to a particular notebook within the examples folder).

• Urban plume scenario demo (as in PartMC):
  
• Dry-Wet Particle Size Equilibration with PartMC and PySDM:
  
• Simulation output processing example (loading from netCDF files using PyPartMC):
  
• Optical properties calculation using external Python package (PyMieScatt):
  
• Cloud parcel example featuring supersaturation-evolution-coupled CCN activation and drop growth:
  

Features

• works on Linux, macOS and Windows (compatibility assured with CI builds)
• hassle-free installation using pip (prior PartMC installation not needed)
• works out of the box on mybinder.org, Google Colab and alike
• ships with a set of examples maintained in a form of Jupyter notebooks
• Pythonic API (but retaining PartMC jargon) incl. Python GC deallocation of Fortran objects
• specification of parameters using native Python datatypes (lists, dicts) in place of PartMC spec files
• code snippets in README depicting how to use PyPartMC from Julia and Matlab (also executed on CI)
• auto-generated API docs on the web
• support for [de]serialization of selected wrapped structures using JSON
• based on unmodified PartMC code
• does not use or require shell or any pre-installed libraries
• aiming at 100% unit test coverage

Usage examples

The listings below depict how the identical task of randomly sampling particles from an aerosol size distribution in PartMC can be done in different programming languages.

For a Fortran equivalent of the Python, Julia and Matlab programs below, see the readme_fortran folder.

Python

import numpy as np

import PyPartMC as ppmc
from PyPartMC import si

aero_data = ppmc.AeroData((
    #      [density, ions in solution, molecular weight, kappa]
    {"OC": [1000 *si.kg/si.m**3, 0, 1e-3 *si.kg/si.mol, 0.001]},
    {"BC": [1800 *si.kg/si.m**3, 0, 1e-3 *si.kg/si.mol, 0]},
))

aero_dist = ppmc.AeroDist(
    aero_data,
    [{
        "cooking": {
            "mass_frac": [{"OC": [1]}],
            "diam_type": "geometric",
            "mode_type": "log_normal",
            "num_conc": 3200 / si.cm**3,
            "geom_mean_diam": 8.64 * si.nm,
            "log10_geom_std_dev": 0.28,
        }
    },
    {
        "diesel": {
            "mass_frac": [{"OC": [0.3]}, {"BC": [0.7]}],
            "diam_type": "geometric",
            "mode_type": "log_normal",
            "num_conc": 2900 / si.cm**3,
            "geom_mean_diam": 50 * si.nm,
            "log10_geom_std_dev": 0.24,
        }
    }],
)

n_part = 100
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source")
aero_state.dist_sample(aero_dist)
print(np.dot(aero_state.masses(), aero_state.num_concs), "# kg/m3")

Julia (using PyCall.jl)

using Pkg
Pkg.add("PyCall")

using PyCall
ppmc = pyimport("PyPartMC")
si = ppmc["si"]

aero_data = ppmc.AeroData((
  #       (density, ions in solution, molecular weight, kappa)
  Dict("OC"=>(1000 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0.001)),
  Dict("BC"=>(1800 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0))
))

aero_dist = ppmc.AeroDist(aero_data, (
  Dict( 
    "cooking" => Dict(
      "mass_frac" => (Dict("OC" => (1,)),),
      "diam_type" => "geometric",
      "mode_type" => "log_normal",
      "num_conc" => 3200 / si.cm^3,
      "geom_mean_diam" => 8.64 * si.nm,
      "log10_geom_std_dev" => .28,
    )
  ),
  Dict( 
    "diesel" => Dict(
      "mass_frac" => (Dict("OC" => (.3,)), Dict("BC" => (.7,))),
      "diam_type" => "geometric",
      "mode_type" => "log_normal",
      "num_conc" => 2900 / si.cm^3,
      "geom_mean_diam" => 50 * si.nm,
      "log10_geom_std_dev" => .24,
    )
  )
))

n_part = 100
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source")
aero_state.dist_sample(aero_dist)
print(aero_state.masses()'aero_state.num_concs, "# kg/m3")

Matlab (using Matlab's built-in Python interface)

notes (see the PyPartMC Matlab CI workflow for an example on how to achieve it on Ubuntu 20):

• Matlab ships with convenience copies of C, C++ and Fortran runtime libraries which are dlopened() by default; one way to make PyPartMC OK with it is to [pip-]install by compiling from source using the very same version of GCC that Matlab borrowed these libraries from (e.g., GCC 9 for Matlab R2022a, etc);
• Matlab needs to use the same Python interpretter/venv as the pip invocation used to install PyPartMC;
• a single-line pybind11_builtins.py file with just pybind11_type=type inside needs to be placed within Matlab's PYTHONPATH to sort out a Matlab-pybind11 incompatibility.

ppmc = py.importlib.import_module('PyPartMC');
si = py.importlib.import_module('PyPartMC').si;

aero_data = ppmc.AeroData(py.tuple({ ...
  py.dict(pyargs("OC", py.tuple({1000 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0.001}))), ...
  py.dict(pyargs("BC", py.tuple({1800 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0}))) ...
}));

aero_dist = ppmc.AeroDist(aero_data, py.tuple({ ...
  py.dict(pyargs( ...
    "cooking", py.dict(pyargs( ...
      "mass_frac", py.tuple({py.dict(pyargs("OC", py.tuple({1})))}), ...
      "diam_type", "geometric", ...
      "mode_type", "log_normal", ...
      "num_conc", 3200 / si.cm^3, ...
      "geom_mean_diam", 8.64 * si.nm, ...
      "log10_geom_std_dev", .28 ...
    )) ...
  )), ...
  py.dict(pyargs( ... 
    "diesel", py.dict(pyargs( ...
      "mass_frac", py.tuple({ ...
        py.dict(pyargs("OC", py.tuple({.3}))), ...
        py.dict(pyargs("BC", py.tuple({.7}))), ...
      }), ...
      "diam_type", "geometric", ...
      "mode_type", "log_normal", ...
      "num_conc", 2900 / si.cm^3, ...
      "geom_mean_diam", 50 * si.nm, ...
      "log10_geom_std_dev", .24 ...
    )) ...
  )) ...
}));

n_part = 100;
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source");
aero_state.dist_sample(aero_dist);
masses = cell(aero_state.masses());
num_concs = cell(aero_state.num_concs);
fprintf('%g # kg/m3\n', dot([masses{:}], [num_concs{:}]))

usage in other projects

PyPartMC is used within the test workflow of the PySDM project.

FAQ

• Q: How to install PyPartMC with MOSAIC enabled?
  A: Installation can be done using pip, however, pip needs to be instructed not to use binary packages available at pypi.org but rather to compile from source (pip will download the source from pip.org), and the path to compiled MOSAIC library needs to be provided at compile-time; the following command should convey it:

MOSAIC_HOME=<<PATH_TO_MOSAIC_LIB>> pip install --force-reinstall --no-binary=PyPartMC PyPartMC

• Q: Why pip install PyPartMC triggers compilation on my brand new Apple machine, while it quickly downloads and installs binary packages when executed on older Macs, Windows or Linux?
  A: We are providing binary wheels on PyPI for Apple-silicon (arm64) machines for selected macOS version made available by Github. In case the macOS version you are using is newer, compilation from source is triggered.

• Q: Why some of the constructors expect data to be passed as lists of single-entry dictionaries instead of multi-element dictionaries?
  A: This is intentional and related with PartMC relying on the order of elements within spec-file input; while Python dictionaries preserve ordering (insertion order), JSON format does not, and we intend to make these data structures safe to be [de]serialized using JSON.

• Q: How to check the version of PartMC that PyPartMC was compiled against?
  A: Version numbers of compile-time dependencies of PyPartMC, including PartMC, can be accessed as follows:

import PyPartMC
PyPartMC.__versions_of_build_time_dependencies__['PartMC']

• Q: Why m4 and perl are required at compile time?
  A: PyPartMC includes parts of netCDF and HDF5 codebases which depend on m4 and perl, respectively, for generating source files before compilation.

Troubleshooting

Common installation issues

error: [Errno 2] No such file or directory: 'cmake'

Try rerunning after installing CMake, e.g., using apt-get install cmake (Ubuntu/Debian), brew install cmake (homebrew on macOS) or using MSYS2 on Windows.

No CMAKE_Fortran_COMPILER could be found.

Try installing a Fortran compiler (e.g., brew reinstall gcc with Homebrew on macOS or using MSYS2 on Windows).

Could not find NC_M4 using the following names: m4, m4.exe

Try installing m4 (e.g., using MSYS2 on Windows).

Credits

PyPartMC:

authors: PyPartMC developers
funding: US Department of Energy Atmospheric System Research programme, Polish National Science Centre
copyright: University of Illinois at Urbana-Champaign
licence: GPL v3

PartMC:

authors: Nicole Riemer, Matthew West, Jeff Curtis et al.
licence: GPL v2 or later