Skip to main content

Tools used to publish Python packages, Docker images and Helm charts for GitHub tag and branch

Project description

Tag Publish

Publishing

The main goals of Tag Publish offer the commands to publish the project, Using a tag, a stabilization branch, a feature branch or a pull request.

When possible it can do a secret-less publishing, if it's not possible the login should be done before the publishing.

See the documentation.

Startup

Set the permissions:

permissions:
  # To publish Docker images on GHCR and on npm.pkg.github.com
  packages: write
  # To publish Python packages using OIDC
  id-token: write
  # To publish Helm charts and send repository dispatch notifications
  contents: write

Install the package in the worklow:

- name: Install tag-publish
  run: pip install tag-publish

Do the publishing:

- name: Publish
  run: tag-publish
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

New version

To create a new minor version you just should run tag-publish-new --version=<version>.

This will create the stabilization branch and will create a new pull request to update the SECURITY.md file and the Renovate configuration.

This will also create the tags for the backport.

You are welcome to run tag-publish-new --help to see what's it's done.

Note that it didn't create a tag, you should do it manually.

To create a patch version you should just create tag.

SECURITY.md

The SECURITY.md file should contain the security policy of the repository, especially the end of support dates.

For compatibility with security.md it should contain an array with at least the columns Version and Supported Until. The Version column will contain the concerned version. The Supported Until will contain the date of end of support dd/mm/yyyy. It can also contain the following sentences:

  • Unsupported: no longer supported => no audit, no rebuild.
  • Best effort: the support is ended, it is still rebuilt and audited, but this can be stopped without any notice.
  • To be defined: not yet released or the date will be set related of another project release date (like for GeoMapFish).

See also GitHub Documentation

Configuration

The configuration file is .github/publish.yaml, the schema is https://raw.githubusercontent.com/camptocamp/tag-publish/<version>/tag_publish/schema.json.

Dry run

Dry run publish: GITHUB_REF=... tag-publish --dry-run ...

Python package to pypi repository

Minimum configuration:

pypi:
  packages:
    - {}

If the file ~/.pypirc didn't exists we will do a login using OpenId Connect (OIDC), see: https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi.

By default the package will be published only on tag, if you want to publish on stabilization branch you should add a versions key with the list of versions you want to publish, that can be: rebuild (specified with --type), version_tag, version_branch, feature_branch, feature_tag (for pull request)

It we have a setup.py file, we will be in legacy mode: When publishing, the version computed from arguments or GITHUB_REF is put in environment variable VERSION, thus you should use it in setup.py, example:

VERSION = os.environ.get("VERSION", "1.0.0")

Also we consider that we use poetry with poetry-dynamic-versioning to manage the version, and poetry-plugin-tweak-dependencies-version to manage the dependencies versions.

Example of configuration:

[tool.poetry-dynamic-versioning]
enable = true
vcs = "git"
pattern = "^(?P<base>\\d+(\\.\\d+)*)"
format-jinja = """
{%- if env.get("VERSION_TYPE") == "version_branch" -%}
{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
{%- elif distance == 0 -%}
{{serialize_pep440(base)}}
{%- else -%}
{{serialize_pep440(bump_version(base), dev=distance)}}
{%- endif -%}
"""

Note that we can access to the environment variables VERSION,VERSION_TYPE and IS_MASTER.

Then by default:

  • Tag with 1.2.3 => release 1.2.3
  • Commit on feature branch just do a validation
  • Commit on master branch after the tag 1.3.0 => release 1.4.0.dev1
  • Commit on 1.3 branch after the tag 1.3.0 => release 1.3.1.dev1

Authentication

If the file ~/.pypirc exists we consider that we ar already logged in also we will do the login with the pypi server with OpenID Connect (OIDC).

The OIDC login is recommended because it didn't needs any additional secrets, but it need some configuration on pypi in the package, see the GitHub Documentation.

The required permissions is id-token: write.

Integration if the package directly in a Docker image

To make it working in the Dockerfile you should have in the poetry stage:

ENV POETRY_DYNAMIC_VERSIONING_BYPASS=dev
RUN poetry export --extras=checks --extras=publish --output=requirements.txt \
    && poetry export --with=dev --output=requirements-dev.txt

And in the run stage

ARG VERSION=dev
RUN --mount=type=cache,target=/root/.cache \
    POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=.

And in the Makefile:

VERSION = $(strip $(shell poetry version --short))

.PHONY: build
build: ## Build the Docker images
    docker build --build-arg=VERSION=$(VERSION) --tag=$(GITHUB_REPOSITORY) .

Docker image to registry

The minimal config is like this:

docker:
  images:
    - name: camptocamp/tag-publish

If you want to use the GitHub token to be logged in on ghcr you should set auto_login to True, the requires the permissions are packages: write.

With that the image initially named camptocamp/tag-publish:latest will be published on GitHub CHCR and on Docker hub.

The full config is like this:

docker:
  auto_login: False
  latest: True
  images:
    - # The base name of the image we want to publish
      name:
  repository:
    <internal_name>:
      # The fqdn name of the server if not Docker hub
      server:
      # List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
      # version_tag, version_branch, feature_branch, feature_tag (for pull request)
      version:
      # List of tags we want to publish interpreted with `format(version=version)`
      # e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
      # (that should be built by the application build) is `latest-lite`, and it will be published
      # with the tag `1.2.3-lite`.
      tags:
      # If your images are published by different jobs you can separate them in different groups
      # and publish them with `tag-publish --group=<group>`
      group:

By default, the last line of the SECURITY.md file will be published (docker) with the tag latest. Set latest to False to disable it.

Use Renovate to trigger a new build instead of the legacy rebuild

If the ci/dpkg-versions.yaml or .github/dpkg-versions.yaml file is present, the package list will be updated on publishing.

The versions will be updated by GHCI application.

Node package to npm repository

Minimum configuration:

node:
  packages:
    - {}

If the repository server is npm.pkg.github.com we will do a login using GITHUB_TOKEN.

To publish on npm.pkg.github.com you requires the permissions are packages: write, and the id-token: write for the provenance.

By default the package will be published only on tag.

HELM

The minimal config is like this:

helm:
  packages:
    - {}

This will publish the helm charts in the current folder using chart releaser.

The artifacts will be attached to a GitHub release, and the index.yaml file will be updated in the gh-pages branch.

The required permission is contents: write.

Create the required gh-pages branch:

git checkout --orphan gh-pages
git reset --hard
git commit --allow-empty -m "Initialize gh-pages branch"
git push origin gh-pages

Dispatch

The minimal config is like this:

dispatch:
  - {}

The required permission is contents: write.

This will create a repository dispatch of type published on own repository with the content e.g.:

{
  "version": "1.2.3",
  "version_type": "version_tag",
  "repository": "camptocamp/tag-publish",
  "items": [
    {
      "type": "docker",
      "image": "camptocamp/tag-publish",
      "repository": "ghcr.io",
      "tag": "1.2.3"
    },
    {
      "type": "pypi",
      "path": "."
    },
    {
      "type": "helm",
      "path": "."
    }
  ]
}

Contributing

Install the pre-commit hooks:

pip install pre-commit
pre-commit install --allow-missing-config

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tag_publish-0.12.0.tar.gz (27.2 kB view details)

Uploaded Source

Built Distribution

tag_publish-0.12.0-py3-none-any.whl (27.6 kB view details)

Uploaded Python 3

File details

Details for the file tag_publish-0.12.0.tar.gz.

File metadata

  • Download URL: tag_publish-0.12.0.tar.gz
  • Upload date:
  • Size: 27.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.13.0

File hashes

Hashes for tag_publish-0.12.0.tar.gz
Algorithm Hash digest
SHA256 686edcca3e556331a3e4b5d6a9a4c439eb453a0178c647efbbb26f90bf7bec53
MD5 f15b004866e8efcc68bfbf466c836121
BLAKE2b-256 2e1cf06c60d817ba8d59a772efe01b78773a5a37d4297af2d779b94264c90fbb

See more details on using hashes here.

File details

Details for the file tag_publish-0.12.0-py3-none-any.whl.

File metadata

  • Download URL: tag_publish-0.12.0-py3-none-any.whl
  • Upload date:
  • Size: 27.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.13.0

File hashes

Hashes for tag_publish-0.12.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c1fe884eb0b32b7dcaaef0da0be87ac67c012589b12b0b0a7de3a18d1871fb2d
MD5 dd4d629eb557b831d2edd0e2d853f8de
BLAKE2b-256 338a1b2eff634201e522569e928c88c8ae18ef3e11829895a0b25d528190e3a3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page