ds-reify

A CLI helper tool for Distributed-Something

Development

Setup Virtual Environment

Setup a python environment using the method of your choice.

Using the builtin venv:

python -m venv <ENV_NAME>
source <ENV_NAME>/bin/activate

Using conda (replace with any python >= 3.8):

conda create -n <ENV_NAME> python=3.8
conda activate <ENV_NAME>

Using whatever else you want, like pyenv.

Install dev tools

Install Poetry

curl -sSL https://install.python-poetry.org/ | python
source ~/.poetry/env

Install Nox

pip install --user --upgrade nox

See this post if you're curious as to why we don't install nox via Poetry.

Install pre-commit

pip install --user --upgrade pre-commit

Let Poetry install the rest from pyproject.toml

poetry install

Testing

Coverage.py is used for test coverage, alongside pytest, via the pytest-cov plugin.

To run the tests directly, in you virtual environment, run pytest --cov.

To let nox run across multiple isolated environments, run nox.

To avoid nox recreating the virtual environments from scratch on each invocation, run nox -r.

Run a specific test with nox -s tests -- tests/test_TESTNAME.

Static analysis

Autoformatting is performed with Black.

Run formatting with nox -s black or specify files/directors with nox -s black -- file1 dir1 ....

Black auto-formatting is not run by default when running nox in isolation, it must be specified.

Flake8 is used for linting. Under the hood, it uses:

• pylint
• pyflakes - invalid python code
  • errors reported as F
• pycodestyle - PEP 8 style checking
  • W for warnings, E for errors
• mccabe - code complexity
  • errors reported as C.
• flake8-black plugin - adherence to Black code style
  • erros reported as BLK.
• flake8-import-order plugin - import grouping and ordering checked against the Google styleguide
  • errors reported as I
• flake8-bugbear plugin - various miscellaneous bugs and design problems
  • likely bugs reported as B
  • opinionated bugs reported as B9
  • B950 replaces E501 for max line length checking (adds tolerance margin of 10%)
• flake8-bandit plugin - uses Bandit to find common security issues
  • issues reported as S
• flake8-annotations plugin - detects absence of type annotations for functions
  • issues reported as ANN

All of these are configured in the .flake8 file.

Run linting with nox -s lint or specify files/directoriess with nox -s lint -- file1 dir1 ....

Import ordering is not auto-formatted although may in the future by migrating to flake8-isort.

Safety is uesd for checking project dependencies against known security violations. For example, insecure-package.

Run it with nox -s safety.

Pre-commit

If you would like to enable the pre-commit hooks, run pre-commit install.

The hooks will run on files changed by the commit in question. To trigger hooks automatically run pre-commit run --all-files.