Python project template

A simple template of Python projects, with a rigid file structure, and predisposition for unit testing and release on PyPi.

Relevant features

• All your project code into a single main package (artifact/)
• All your project tests into a single test package (test/)
• Unit testing support via unittest
• Automatic testing on all branches via GitHub Actions
• Semi-automatic versioning via Git
• Packaging support via setuptools
• Automatic release on PyPi via GitHub Actions and semantic-release
• Automatic dependencies updates via Renovate

Project structure

Overview:

<root directory>
├── artifact/             # main package (should be named after your project)
│   ├── __init__.py         # python package marker
│   └── __main__.py         # application entry point
├── tests/                  # test package (should contain unit tests)
├── .github/                # configuration of GitHub CI
│   └── workflows/          # configuration of GitHub Workflows
│       ├── check.yml       # runs tests on multiple OS and versions of Python
│       └── deploy.yml      # if check succeeds, and the current branch is one of {main, master}, triggers automatic releas on PyPi
├── LICENSE                 # license file (Apache 2.0 by default)
├── pyproject.toml          # project configuration file as prescribed by Poetry
├── renovate.json           # configuration of Renovate bot, for automatic dependency updates
├── requirements.txt        # only declares a dependency on Poetry. DO NOT EDIT THIS FILE
└── release.config.js       # script to release on PyPi, and GitHub via semantic-release

TODO-list for template usage

1. Use this template to create a new GitHub repository, say artifact

   • this name will also be used to identify the package on PyPi
     • so, we suggest choosing a name which has not been used on PyPi, yet
     • we also suggest choosing a name which is a valid Python package name (i.e. using_snake_case)

2. Clone the artifact repository

3. Open a shell into your local artifact directory and run

   ./rename-template.sh artifact
   

   This will coherently rename the template's project name with the one chosen by you (i.e. artifact, in this example)

   • Remark: this step is now automatic thanks to the init.yml workflow which is triggered when using this template to create a new repository

4. Commit & push

5. Ensure you like the Apache 2.0 License. If you don't, change the content of the LICENSE file

6. Ensure the versions-range of Python reported in pyproject.toml fits the versions you want to support

   • currently defaults to >= 3.9
   • if you change this, please also change the versions of Python tests should be run on in CI, by looking the file .github/workflows/check.yml

7. Check the Python version and OS tests should be run on in CI, by looking the file .github/workflows/check.yml

8. Add your runtime, development, and build dependencies to pyproject.toml

9. Check the other metadata in pyproject.toml

10. Change the assignee for pull-requests for automatic dependency updates by editing renovate.json

    • currently defaults to @gciatto

11. Add your PYPI_TOKEN token as secrets of the GitHub repository

    • this may require you to register on PyPi first
    • Note: PyPI no longer allows authentication with a username and password. Please follow these steps instead:
      1. Authenticate on PyPi
      2. Go to your account settings and generate a new API token
      3. Add the API Token to the PYPI_PASSWORD secret
      4. Add the string __token__ to the PYPI_USERNAME secret

12. Generate a GitHub token and add it as a secret of the GitHub repository, named RELEASE_TOKEN

    • cf. https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic
    • the token must allow pushing to the repository

13. Put your main (resp. test) code in artifact/ (resp. test/)

How to do stuff

Restore dev dependencies

1. Install Poetry if you don't have it yet

   pip install -r requirements.txt
   

2. Install the project's dependencies

   poetry install
   

Run Tests

Execute the test suite using pytest:

poetry run poe test

Run Tests with Coverage

Execute the test suite with coverage reporting:

poetry run poe coverage

and generate a report with poe coverage-report or poe coverage-html

Run Static Checks

Perform static code analysis using both mypy and ruff:

poetry run poe static-checks

Format Code

Format your code using ruff:

poetry run poe format

Note: you can enter a Poetry shell via poetry shell to avoid prefixing commands with poetry run.

Tests are automatically run in CI, on all pushes on all branches. There, tests are executed on multiple OS (Win, Mac, Ubuntu) and on multiple Python versions.

Run your code as an application

This will execute the __main__.py file in the artifact package:

poetry run python -m artifact

the latter is possible because of the script defined in the pyproject.toml file.

Release a new version on PyPi

New versions are automatically released on PyPi via GitHub Actions, when a push is made on the main or master branch.

The version number is updated automatically by the semantic-release tool, which uses the commit messages to infer the type of the release (major, minor, patch).

It is paramount that the commit messages follow the Conventional Commits specification, in order for semantic-release to compute version numbers correctly.

Automatic updates of dependencies (via Renovate)

The project is configured to use Renovate to automatically open pull-requests to update dependencies declared in pyproject.toml.

By default, Renovate will assign such pull-requests to the user who created the repository from this template.

If the project has tests (which is the case for this template), Renovate will only merge such pull-requests if all tests pass.

When some test fails, Renovate will leave a comment on the pull-request, so that you can fix the issue manually.

To make Renovate work, you need to enable it for your repository. To do so, please follow the instruction at https://docs.renovatebot.com/getting-started/installing-onboarding/#hosted-githubcom-app

Finally, please remember to enable PR auto-merging in your repository settings, otherwise Renovate will not be able to merge the pull-requests it opens, even if all tests pass. To do so, please follow the instructions available here.

Notice that the combination between Renovate, and Semantic Release may lead to a number of releases being created automatically.