Python implementations of REALM models
Project description
The pyrealm
package
These are development notes for the package, user documentation can be found at:
https://pyrealm.readthedocs.io/
Overview
This a Python 3 package intended to provide a common framework for a number of related models of plant productivity, growth and demography.
Code development
The codebase is developed in git
with a repository at:
https://github.com/davidorme/pyrealm
It uses the git flow
model for development and release. Briefly:
- All code development should happen on the general
develop
branch or on specificfeature/feature_name
branches. - Candidate release versions should be made on specific
release/x.y.z
branches and these are then committed to themaster
branch only after final checking. - The
master
branch should only ever contain commits representing new release versions - do not work on themaster
branch.
Continuous integration
The project uses continuous integration on the Travis platform to check that the package is building correctly as changes are committed to Github. The status of builds can be seen at:
https://travis-ci.com/github/davidorme/pyrealm
Documentation
The pyrealm
package is documented using sphinx
, with source material in the
source
directory.
The documentation in source
uses Myst Markdown
rather than the standard sphinx
reStructuredText (.rst
) format. This is
because the documentation uses the myst_nb
extension to sphinx
that supports
running documentation as a Jupyter notebook: the built documentation includes
examples of running code and output plots to demonstrate the use and behaviour
of the package.
The sphinx
configuration includes the sphinx.ext.mathjax
extension to support mathematical notation. This has been configured to also
load the mhchem
extension, supporting the rendering of chemical notation.
Docstrings
The module codes uses docstrings written in the
Google style.
Unlike the main documentation pages, the docstrings in code are written using
reStructuredText because the autodoc
functions in sphinx
currently rely on rst
inputs. This allows the function documentation to be stored alongside the code
and included simply into the documentation.
Building the documentation
Additional python packages given in source/requirements.txt
are needed
to build the documentation. To actually build the documentation, use
make
in the package root, which will use the Makefile
created by
sphinx-quickstart
.
make html
Online documentation
TODO - change this to github deployment?
The documentation for the package is hosted at:
https://pyrealm.readthedocs.io/en/develop/pmodel.html
This has been configured to build commits to the master
branch, which should
generate version specific sets of documentation.
Referencing
The documentation uses the sphinxcontrib-bibtex
package to support citations.
This uses Latex like citation keys in the documentation to insert references and
build a bibliography. The reference library in source/refs.bib
needs to be
kept up to date with the literature for the project.
At present, that package uses a rather ugly citation style. I'm hoping the
sphinx_astrorefs
package will help out, but there is currently an issue
getting that package to load.
Testing
Developer installation
Use the local directory as an editable installation of the package
pip install -e .
Using doctest
The package docstrings contain doctest
examples of code use. These are
intended to demonstrate use and to validate a reference set of inputs against
expected outputs. They do not provide extensive unit testing! To run the
docstring tests, use:
python -m doctest pyrealm/pmodel.py
python -m doctest pyrealm/*.py
For doctest
on warnings, see the example for pyrealm.utilities.convert_rh_to_vpd
which redirects the stderr to stdout to allow for the warning text to be
included in the doctest.
Using pytest
The test
directory contains pytest
modules to provide greater testing of
different input combinations (scalars and arrays) and to check errors are
raised correctly.
pytest
Reference values for testing
The sources of the reference inputs and outputs are:
pmodel
module:
Benjamin Stocker's rpmodel
implementation of the P-model in R. The test
directory contains a YAML
file of inputs (test_inputs.yaml
) and an R
script (test_output_rpmodel.R
)
that are used to generate a larger YAML file (test_outputs_rpmodel.R
) that
are loaded and validated against {mod}pyrealm.pmodel
by test_pmodel.py
.
Continuous Integration
Git flow
Configure git
It is easier if git
is configured to push new tags along with commits. This
essentially just means that new releases can be sent with a single commit,
which is simpler and saves Travis from building both the the code commit and
then the tagged version. This only needs to be set once.
set git config --global push.followTags true
Using git-flow and travis
Use git flow to create a release
git flow release start 0.3.0
and then bump the version number in version.py
.
Check the package builds and installs locally:
python setup.py sdist bdist_wheel
Commit the version number change and then publish the branch:
git commit -m "Version bump" pyrealm/version.py
git flow release publish x.y.z
to get the branch onto the origin repository and hence into Travis.
Use twine to check it passes onto testpypi
We are using twine
to publish versions to PyPi, and using the testpypi
sandbox to check release candidates are accepted. This needs an account
for both pypi and testpypi.
Remembering to change the version number, use twine
to upload the built
versions to the testpypi
site:
twine upload -r testpypi dist/*x.y.z*
Check the documentation builds
Log in to:
which is the admin site controlling the build process. From the Versions
tab, activate the release/x.y.z
branch and wait for it to build. Check
the Builds tab to see that it has built successfully and maybe check
updates! If it has built succesfully, then go back to the Versions tab
and deactivate and hide the branch.
Success
Once all seems well, the next step is to finish the release, which merges
changes into develop
and into a tagged commit on master
. You then need
to checkout the master branch and push the new version and tag to GitHub.
git flow release finish x.y.z
git checkout master
git push
PyPi upload
To upload the new version to the main PyPi site, run the build process again
in the master
branch to get the release builds:
python setup.py sdist bdist_wheel
And then release the distribution using twine
for use via pip
- this time
not using the testpypi
sandbox.
twine upload dist/*x.y.z*
Now:
- switch back to
develop
!
git checkout develop
- Bump the version number to add
.post9000
to show the code is in development again.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.