Python implementations of REALM models
Project description
The pyrealm
package
These are development notes for the package, user documentation can be found at:
Overview
This a Python 3 package intended to provide a common framework for a number of related models of plant productivity, growth and demography.
Code development
Package managment
The package uses poetry
to manage dependencies, generate virtual environments for code
development and then for package building and publication. You will need to install
poetry
to develop the code.
Source code management
The codebase is developed in git
with a repository at:
It uses the git flow
model for development and release. Briefly:
- All code development should happen specific
feature/feature_name
branches. - Pull requests (PR) should be made to merge feature branches into the
develop
branch. - Candidate release versions should be made on specific
release/x.y.z
branches and these are then committed to themaster
branch only after final checking. - The
master
branch should only ever contain commits representing new release versions - do not work on themaster
branch.
Both the develop
and master
branches are protected on GitHub to avoid accidents!
Code quality
The project uses the pre-commit
tool to enforce code quality. The configuration file
.pre-commit-config.yaml
shows the details of the tool chain, but isort
, black
,
flake8
and markdownlint
are used to maintain code quality. You will need to install
pre-commit
to develop package code, and each PR must pass the same set of checks.
Code testing
Using doctest
The package docstrings contain doctest
examples of code use. These are intended to
demonstrate use and to validate a reference set of inputs against expected outputs. They
do not provide extensive unit testing! To run the docstring tests, use:
python -m doctest pyrealm/pmodel.py
python -m doctest pyrealm/*.py
For doctest
on warnings, see the example for pyrealm.utilities.convert_rh_to_vpd
which redirects the stderr to stdout to allow for the warning text to be included in the
doctest.
Using pytest
The test
directory contains pytest
modules to provide greater testing of different
input combinations and to check errors are raised correctly. These are the main tests
that the package is behaving as expected.
pytest
Continuous integration
The project uses continuous integration via GitHub Actions to check that the package is
building correctly and that both doctest
and pytest
tests are passing. The status of
builds can be seen at:
Documentation
The pyrealm
package is documented using sphinx
, with source material in the
docs/source
directory.
The documentation in source
uses Myst
Markdown rather than the standard
sphinx
reStructuredText (.rst
) format. This is because the documentation uses the
myst_nb
extension to sphinx
that supports running documentation as a Jupyter
notebook: the built documentation includes examples of running code and output plots to
demonstrate the use and behaviour of the package.
The sphinx
configuration includes the sphinx.ext.mathjax
extension to support
mathematical notation. This has been configured to also load the mhchem
extension,
supporting the rendering of chemical notation.
Docstrings
The module codes uses docstrings written in the Google
style.
Unlike the main documentation pages, the docstrings in code are written using
reStructuredText because the autodoc
functions in sphinx
currently rely on rst
inputs. This allows the function documentation to be stored alongside the code and
included simply into the documentation.
Building the documentation
Additional python packages given in docs/source/requirements.txt
are needed to build
the documentation. To actually build the documentation, use make
in the package root,
which will use the Makefile
created by sphinx-quickstart
.
cd docs
make html
Online documentation
The documentation for the package is hosted at:
This has been configured to build commits to the master
branch, which should
generate version specific sets of documentation.
Referencing
The documentation uses the sphinxcontrib-bibtex
package to support citations.
This uses Latex like citation keys in the documentation to insert references and
build a bibliography. The reference library in source/refs.bib
needs to be
kept up to date with the literature for the project. The sphinx_astrorefs
package is
used to provide an Author Date style citation format.
Release process
Releasing a new version of the package follows the flow below:
- Create a
release
branch fromdevelop
containing the new release code. - Check that this branch builds correctly, that the documentation builds correctly and
that the package publishes to the
test-pypi
repository. - When all is well, create pull requests on GitHub to merge the
release
branch into bothdevelop
andmaster
, along with a version tag for the release. - Once you have updated your local repository, then the tag can be used to build and publish the final version to the main PyPi site.
It is easier if git
is configured to push new tags along with commits. This
essentially just means that new releases can be sent with a single commit. This only
needs to be set once.
set git config --global push.followTags true
Create the release branch
Using git flow
commands as an example to create a new release:
git flow release start new_release
Obviously, use something specific, not new_release
! Ideally, you would do a dry run of
the next step and use the version - but it should be fairly obvious what this will be!
The poetry version
command can then be used to bump the version number. Note that the
command needs a 'bump rule', which sets which part of the semantic version number to
increment (major
, minor
or patch
). For example:
poetry version patch
This updates pyproject.toml
. At present, the package is set up so that you also have
to update the version number in pyrealm/version.py
to match manually.
Publish and test the release branch
With those changes committed, publish the release branch:
git commit -m "Version bump" pyrealm/version.py
git flow release publish new_release
The GitHub Actions will then ensure that the code passes quality assurance and then runs the test suites on a range of Python versions and operating systems.
Check package publication
The sdist
and wheel
builds for the package can then be built locally using poetry
poetry build
The first time this is run, poetry
needs to be configured to add the Test PyPi
repository and an API token from that site. Note that accounts are not shared between
the Test and main PyPi sites: the API token for test-pypi
is different from
pypi
and you have to log in to each system separately and generate a token on each.
poetry config repositories.test-pypi https://test.pypi.org/legacy/
poetry config pypi-token.test-pypi <your-token>
The built packages can then be published to test-pypi
using:
poetry publish -r test-pypi
Check the documentation builds
Log in to:
which is the admin site controlling the build process. From the Versions tab, activate
the release/new_release
branch and wait for it to build. Check the Builds tab to see
that it has built successfully and maybe check updates! If it has built succesfully, do
check pages to make sure that page code has executed successfully, and then go back to
the Versions tab and deactivate and hide the branch.
Create pull requests into master
and develop
If all is well, then two PRs need to be made on GitHub:
- The
release
branch intomaster
, to bring all commits since the last release and any fixes on release intomaster
. - The
release
branch intodevelop
, to bring anyrelease
fixes back intodevelop
.
Once both of those have been merged, the feature
branch can be deleted.
Tag, build and publish
Once the origin
repository is merged, then use git pull
to bring develop
and
master
up to date on a local repo. Then, create a tag using the release version.
git checkout master
git tag <version>
git push --tags
The final commit on master
is now tagged with the release version. You can add tags on
the GitHub website, but only by using the GitHub release system and we are using PyPi to
distribute package releases.
Before publishing a package to the main PyPi site for the first time, you need to set an API token for PyPi.
poetry config pypi-token.pypi <your-token>
And now you can build the packages from master
and publish.
poetry build
poetry publish
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.