Skip to main content

pypeline - Define Your CI/CD Pipeline Once, Run It Anywhere

CI Status Documentation Status Test coverage percentage

Poetry ruff pre-commit

PyPI Version Supported Python versions License

Pypeline lets you define your build, test, and deployment pipeline in a single YAML file and run it consistently across your local development environment and any CI/CD platform (GitHub Actions, Jenkins, etc.). No more platform-specific configurations – write once, run anywhere.

Key Features

  • Unified Pipeline Definition: Users can define their entire pipeline in a single YAML file, eliminating the need to switch between different syntaxes and configurations for different CI/CD tools.

  • Extensibility: Pypeline supports execution steps defined not only through installed Python packages but also from local scripts.

  • Execution Context: Allow sharing information and state between steps. Each step in the pipeline receives an execution context that can be updated during step execution.

  • Dependency Handling: Every step can register its dependencies and will only be scheduled if anything has changed.

  • Use as a Library: PipelineConfig and PipelineLoader[T] are generic — define your own step base class and use pypeline's infrastructure (config loading, step resolution, shared context) to build custom pipelines for any domain, not just CI/CD. See Use Pypeline as a Library.

Installation

Use pipx (or your favorite package manager) to install and run it in an isolated environment:

pipx install pypeline-runner

This will install the pypeline command globally, which you can use to run your pipelines.

[!NOTE] The Python package is called pypeline-runner because the name pypeline was already taken on PyPI. The command-line interface is pypeline.

Documentation: pypeline-runner.readthedocs.io

Quick Start

pipx install pypeline-runner
pypeline init --project-dir my-pipeline
cd my-pipeline
pypeline run

Walkthrough: Getting Started with Pypeline

To get started run the init command to create a sample project:

pypeline init --project-dir my-pipeline

The example project pipeline is defined in the pipeline.yaml file.

pipeline:
  - step: CreateVEnv
    module: pypeline.steps.create_venv
    config:
      bootstrap_script: .bootstrap/bootstrap.py
  - step: WestInstall
    module: pypeline.steps.west_install
    description: Download external modules
  - step: MyStep
    file: steps/my_step.py
    description: Run a custom script

This pipeline consists of three steps:

  • CreateVEnv: This is a built-in step that creates a Python virtual environment.
  • WestInstall: This is a built-in step that downloads external modules using the west tool.
  • MyStep: This is a custom step that runs a script defined in the steps/my_step.py file.

You can run the pipeline using the run command:

pypeline run --project-dir my-pipeline

Contributing

The project uses Poetry for dependencies management and packaging. You can set up the development environment using one of the following methods:

Option 1: Using Poetry directly (minimal setup)

poetry install

Option 2: Using pypeline-runner (runs the full pipeline including tests and checks)

pipx install pypeline-runner
pypeline run

To execute the test suite, call pytest inside Poetry's virtual environment via poetry run:

poetry run pytest

For those using VS Code there are tasks defined for the most common commands:

  • run tests
  • run all checks configured for pre-commit
  • generate documentation

See the .vscode/tasks.json for more details.

This repository uses commitlint for checking if the commit message meets the conventional commit format.

Credits

This package was created with Copier and the browniebroke/pypackage-template project template.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pypeline_runner-1.37.0.tar.gz (40.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pypeline_runner-1.37.0-py3-none-any.whl (48.6 kB view details)

Uploaded Python 3

File details

Details for the file pypeline_runner-1.37.0.tar.gz.

File metadata

  • Download URL: pypeline_runner-1.37.0.tar.gz
  • Upload date:
  • Size: 40.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pypeline_runner-1.37.0.tar.gz
Algorithm Hash digest
SHA256 0170e097b0c18189d9371f6be179ac7fc31075d1597ad64181d3889e1e076138
MD5 633a6d16658f30f4bac77cbc1005425e
BLAKE2b-256 c43ab1569f630acdef916f54ec4601531851e60503c7b98784e84449f63d4888

See more details on using hashes here.

Provenance

The following attestation bundles were made for pypeline_runner-1.37.0.tar.gz:

Publisher: ci.yml on cuinixam/pypeline

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pypeline_runner-1.37.0-py3-none-any.whl.

File metadata

File hashes

Hashes for pypeline_runner-1.37.0-py3-none-any.whl
Algorithm Hash digest
SHA256 cecb88bd6b04225c84849f76bfbe4669387bd82080e4596c5da0a59e9f1c023c
MD5 ea207e63cf2cf1ee390b3c417b549939
BLAKE2b-256 0c2fa49296396e59b5f84694c05a9f3c28ab0f1f83b78e0d6189587e44941bbf

See more details on using hashes here.

Provenance

The following attestation bundles were made for pypeline_runner-1.37.0-py3-none-any.whl:

Publisher: ci.yml on cuinixam/pypeline

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

1.37.0 This release

2 files

1.36.0

2 files

1.35.1

2 files

1.35.0

2 files

1.34.0

2 files

1.33.0

2 files

1.32.0

2 files

1.31.0

2 files

1.30.0

2 files

1.29.2

2 files

1.29.1

2 files

1.29.0

2 files

1.28.0

2 files

1.27.0

2 files

1.26.2

2 files

1.26.1

2 files

1.26.0

2 files

1.25.0

2 files

1.24.1

2 files

1.24.0

2 files

1.23.2

2 files

1.23.1

2 files

1.23.0

2 files

1.22.0

2 files

1.21.1

2 files

1.21.0

2 files

1.20.0

2 files

1.19.1

2 files

1.19.0

2 files

1.18.2

2 files

1.18.1

2 files

1.18.0

2 files

1.17.0

2 files

1.16.1

2 files

1.16.0

2 files

1.15.1

2 files

1.15.0

2 files

1.14.0

2 files

1.13.0

2 files

1.12.0

2 files

1.11.0

2 files

1.10.0

2 files

1.9.1

2 files

1.9.0

2 files

1.8.1

2 files

1.8.0

2 files

1.7.0

2 files

1.6.0

2 files

1.5.0

2 files

1.4.0

2 files

1.3.0

2 files

1.2.0

2 files

1.1.0

2 files

1.0.0

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page