universal-build 0.2.6

Universal build utilities for containerized build pipelines.

universal-build

Universal build utilities for containerized build pipelines.

Getting Started • Features • Documentation • Support • Contribution • Changelog

WIP: This project is still an alpha version and not ready for general usage.

Universal Build allows to implement your build and release pipeline with Python scripts once and run it either on your local machine, in a containerized environment via Act, or automated via Github Actions. It supports a monorepo or polyrepo setup and can be used with any programming language or technology. It also provides a full release pipeline for automated releases with changelog generation.

Highlights

• 🧰  Build utilities for Python, Docker, React & MkDocs
• 🔗  Predefined Github Action Workflows for CI & CD
• 🛠  Integrated with devcontainer for containerized development.
• 🐳  Run builds locally, containerized via Act, or on Github Actions

Getting Started

Installation

Requirements: Python 3.6+.

pip install universal-build

Usage

To make use of universal build for your project, create a build script with the name build.py in your project root. The example below is for a single yarn-based webapp component:

from universal_build import build_utils

args = build_utils.get_sanitized_arguments()

version = args.get(build_utils.FLAG_VERSION)

if args.get(build_utils.FLAG_MAKE):
    build_utils.log("Build the componet:")
    build_utils.run("yarn build", exit_on_error=True)

if args.get(build_utils.FLAG_CHECK):
    build_utils.log("Run linters and style checks:")
    build_utils.run("yarn run lint:js", exit_on_error=True)
    build_utils.run("yarn run lint:css", exit_on_error=True)

if args.get(build_utils.FLAG_TEST):
    build_utils.log("Test the component:")
    build_utils.run("yarn test", exit_on_error=True)

if args.get(build_utils.FLAGE_RELEASE):
    build_utils.log("Release the component:")
    # TODO: release the component to npm with version

Next, copy the build and release-pipeline workflows into your repo into the .github/workflows folder. Once you have pushed the build and release pipelines, you have the following options to trigger your build pipeline:

On your local machine via the build script (you need to have all dependencies for the build installed):

python build.py --make --check --test

In a containerized environment on your local machine via Act:

act -b -s BUILD_ARGS="--check --make --test" -j build

On Github via Github Actions: In the Github UI, go to Actions -> select build-pipeline -> select Run Workflow and provide the build arguments, e.g. --check --make --test. You build pipeline will also run via Github Actions automatically on any push to your repository.

You can find more information on the release pipeline or other features in the features section.

Support & Feedback

This project is maintained by Benjamin Räthlein, Lukas Masuch, and Jan Kalkan. Please understand that we won't be able to provide individual support via email. We also believe that help is much more valuable if it's shared publicly so that more people can benefit from it.

Type	Channel
🚨  Bug Reports
🎁  Feature Requests
👩‍💻  Usage Questions
🗯  General Discussion
❓  Other Requests

Features

Automated Build Pipeline • Automated Release Pipeline • Containerized Development • MkDocs Utilities • Python Utilities • Docker Utilities

Automated Build Pipeline

TBD

Automated Release Pipeline

TBD

Containerized Development

TBD

Simplified Versioning

TBD

MkDocs Utilities

TBD

Python Utilities

TBD

Docker Utilities

TBD

Documentation

Build Script CLI

Any build script that utilizes the build_utils.get_sanitized_arguments() method to parse the CLI arguments can be executed with the following options:

python build.py [OPTIONS]

Options:

• --make: Make/compile/package all artifacts.
• --test: Run unit and integration tests.
• --check: Run linting and style checks.
• --release: Release all artifacts (e.g. to registries like DockerHub or NPM).
• --run: Run the component in development mode (e.g. dev server).
• --version VERSION: Version of the build (MAJOR.MINOR.PATCH-TAG).
• --force: Ignore all enforcements and warnings.
• --skip-path SKIP_PATH: Skips the build phases for all (sub)paths provided here.
• --test-marker TEST_MARKER: Provide custom markers for testing. The default marker for slow tests is slow.
• -h, --help: Show the help message and exit.

Default Flags

At its core, universal-build will parse all arguments provided to the build script via build_utils.get_sanitized_arguments() and returns a sanitized and augmented list of arguments. Those arguments are the building blocks for your build script. You can utilize those arguments in whatever way to like. Here is an example on how to use those arguments in a build.py script:

from universal_build import build_utils

args = build_utils.get_sanitized_arguments()

version = args.get(build_utils.FLAG_VERSION)

if args.get(build_utils.FLAG_MAKE):
  # Run all relevant build commands.

if args.get(build_utils.FLAG_TEST):
  # Run all relevant commands for testing
  test_markers = args.get(build_utils.FLAG_TEST_MARKER)
  if "slow" in test_markers:
    # Run additional slow tests.

The following list contains all of the default flags currently supported by universal-build:

Flag	Type	Description
FLAG_MAKE	bool	Build/compile/package all artifacts.
FLAG_CHECK	bool	Run linting and style checks.
FLAG_TEST	bool	Run unit and integration tests.
FLAG_RELEASE	bool	Release all artifacts (e.g. to registries like DockerHub or NPM).
FLAG_RUN	bool	Run the component in development mode (e.g. dev server).
FLAG_FORCE	bool	Ignore all enforcements and warnings.
FLAG_VERSION	str	Semantic version for the build. If not provided via CLI arguments, a valid dev version will be automatically calculated.
FLAG_TEST_MARKER	List[str]	Custom markers for testing. Can be used to skip or execute certain tests.

API Reference

In addition to argument parsing capabilities, universal-build also contains a variety of utility functions to make building complex projects with different technologies easy. You can find all utilities in the Python API documentation here.

Contributors

Contribution

• Pull requests are encouraged and always welcome. Read our contribution guidelines and check out help-wanted issues.
• Submit Github issues for any feature request and enhancement, bugs, or documentation problems.
• By participating in this project, you agree to abide by its Code of Conduct.
• The development section below contains information on how to build and test the project after you have implemented some changes.

Development

Requirements: Docker and Act are required to be installed on your machine to execute the build process.

To simplify the process of building this project from scratch, we provide build-scripts that run all necessary steps (build, check, test, and release) within a containerized environment. To build and test your changes, execute the following command in the project root folder:

act -b -j build

Refer to our contribution guides for more detailed information on our build scripts and development process.

---

Licensed MIT. Created and maintained with ❤️  by developers from Berlin.