project-structure 0.8.27

Project structure foundation or template, CLI console scripts.

This repository is meant to be used as a minimal, yet opinionated baseline for Python software projects. It includes:

• Basic Python “distribution”/project metadata

• Command-line console script with sub-command boilerplate

• A Makefile for local development build, test and maintenance tasks

• Docker container images for users and development in which tests are run

• A Makefile target to format all Python code, including using Black for style

• A kitchen sink linter configuration for Prospector that runs all available Python code checks

• A tox.ini configuration for Tox to run all tests and linters in multiple Python versions, including some checks not provided by Prospector.

• VCS hooks to enforce conventional commits and successful build and test on commit and push, and release notes on push

• Targets/recipes in the Makefile to automate releases controlled by conventional commits and end-user oriented release notes by Towncrier

• Targets/recipes in the Makefile to automate upgrading requirements and dependencies

• Makefile recipes/targets used for both local development and on CI/CD platforms

• A GitLab CI/CD pipeline integrating those CI/CD recipes/targets

• A GitHub Actions workflow/pipeline integrating those CI/CD recipes/targets

The intended use is to add this repository as a VCS remote for your project. Thus developers can merge changes from this repository as we make changes related to Python project structure and tooling. As we add structure specific to certain types of projects (e.g. CLI scripts, web development, etc.), frameworks (e.g. Flask, Pyramid, Django, etc.), libraries and such, branches will be used for each such variation such that structure common to different variations can be merged back into the branches for those specific variations.

Table of Contents

• Template Usage

• Installation

  • Local/Native Installation

  • Docker Container Image Installation

• Usage

• Contributing

• Motivation

Template Usage

This is a rough guide to applying this project template to your project. This is not thoroughly tested as such tests would be so meta as to be extremely wasteful of developer time to create and maintain. So report any issues you have or better yet figure it out and submit a PR with corrections to this section.

1. Choose the right branch to use:

   Is your project a CLI utility? A web application? Which project hosting provider and/or CI/CD platform will you use? Choose the appropriate branch for your project:

   • dist:

     Basic Python distribution with build, tests, linters, code formatting and release publishing from local developer checkouts.

   • cli:

     The above plus support for project’s that provide an executable CLI.

   • docker:

     The dist branch plus Docker containers for both development and end-users/deployments.

   • ci:

     The above plus GitLab CI/CD pipelines that run tests and linters as CI and publish releases from develop and main as CD.

   • ci-cli:

     The above plus the cli branch.

   • etc.

   Do not use the develop or main branches in your project as those branches are used to test the CI/CD automatic releases process and as such contain bumped versions, release notes, and other release artifacts that shouldn’t be merged into real projects.

2. Reconcile VCS history:

   If starting a fresh project:

   $ git clone --origin "template" --branch "${TEMPLATE_BRANCH:?}" \
   "https://gitlab.com/rpatterson/project-structure.git" "./foo-project"
   $ cd "./foo-project"
   $ git config remote.template.tagOpt --no-tags
   $ git remote add "origin" "[e-mail]:foo-username/foo-project.git"
   $ git switch -C "main" --track "origin/main"

   If merging into an existing project:

   $ git remote add "template" \
   "https://gitlab.com/rpatterson/project-structure.git"
   $ git config remote.template.tagOpt --no-tags
   $ git merge --allow-unrelated-histories "template/${TEMPLATE_BRANCH:?}"

3. Rename file and directory paths derived from the project name:

   $ git ls-files | grep -iE 'python.?project.?structure'

4. Rename strings derived from the project name and template author identity in project files:

   $ git grep -iE 'python.?project.?structure|ross|Patterson'

5. Examine # TEMPLATE: comments and change as appropriate:

   These are the bits that need the developer’s attention and reasoning to take the correct action. So read the comments and address them with care and attention:

   $ git grep "TEMPLATE"

Finally, remove this section from this ./README.rst and update the rest of it’s content as appropriate for your project. As fixes and features are added to the upstream template, you can merge them into your project and repeat steps 3-5 above as needed.

This template publishes pre-releases on all pushes to the develop branch and final releases on all pushes to the main branch. Project owners may decide which types of changes should go through pre-release before final release and which types of changes should go straight to final release. For example they may decide that:

• Contributions from those who are not maintainers or owners should be merged into develop. See the ./CONTRIBUTING.rst file for such an example public contributions policy and workflow.

• Fixes for bugs in final releases may be committed to a branch off of main and, after passing all tests and checks, merged back into main to publish final releases immediately.

• Routine version upgrades for security updates may also be merged to main as above for bug fixes.

Installation

Local/Native Installation

Install using any tool for installing standard Python 3 distributions such as pip:

$ pip3 install --user project-structure

Optional shell tab completion is available via argcomplete.

Optional shell tab completion is available via argcomplete.

Docker Container Image Installation

The recommended way to use the Docker container image is via Docker Compose. See the example ./docker-compose.yml file for an example configuration. Once you have your configuration, you can create and run the container:

$ docker compose up

Alternatively, you make use the image directly. Pull the Docker image:

$ docker pull "registry.gitlab.com/rpatterson/project-structure"

And then use the image to create and run a container:

$ docker run --rm -it "registry.gitlab.com/rpatterson/project-structure" ...

Images variant tags are published for the Python version, branch, and major/minor versions so that users can control when they get new images over time, e.g. registry.gitlab.com/rpatterson/project-structure:py310-main. The canonical Python version is 3.10 which is the version used in tags without py###, e.g. registry.gitlab.com/rpatterson/project-structure:main. Pre-releases are from develop and final releases are from main which is also the default for tags without a branch, e.g. registry.gitlab.com/rpatterson/project-structure:py310. The major/minor version tags are only applied to the final release images and without the corresponding main branch tag, e.g. registry.gitlab.com/rpatterson/project-structure:py310-v0.8.

Multi-platform Docker images are published containing images for the following platforms or architectures in the Python 3.10 py310 variant:

• linux/amd64

• linux/arm64

• linux/arm/v7

Usage

See the command-line help for details on options and arguments:

$ project-structure --help
usage: project-structure [-h]

project structure foundation or template, top-level package.

Pptional arguments:
  -h, --help  show this help message and exit

If using the Docker container image, the container can be run from the command-line as well:

$ docker compose run "project-structure" project-structure --help
usage: project-structure [-h]

Project structure foundation or template, top-level package.

optional arguments:
  -h, --help  show this help message and exit

Contributing

NOTE: This project is hosted on GitLab. There’s a mirror on GitHub but please use GitLab for reporting issues, submitting PRs/MRs and any other development or maintenance activity.

See the ./CONTRIBUTING.rst file for more details on how to get started with development.

Motivation

There are many other Python project templates so why make another? I’ve been doing Python development since 1998, so I’ve had plenty of time to develop plenty of opinions of my own.

What I want in a template is complete tooling (e.g. test coverage, linting, formatting, CI/CD, etc.) but minimal dependencies, structure, and opinion beyond complete tooling (e.g. some non-Python build/task system, structure for frameworks/libraries not necessarily being used, etc.). I couldn’t find a template that manages that balance so here we are.

I also find it hard to discern from other templates why they made what choices the did. As such, I also use this template as a way to try out various different options in the Python development world and evaluate them for myself. You can learn about my findings and the reasons the choices I’ve made in the commit history.

Most importantly, however, I’ve never found a satisfactory approach to keeping project structure up to date over time. So the primary motivation is to use this repository as a remote from which we can merge structure updates over the life of projects using the template.