Skip to main content

A library that helps execute pipeline of tasks using filters and simple composition

Project description

step-exec-lib

build codecov PyPI Version Python Versions Apache License

A simple library to easily orchestrate a set of Steps into a filtrable pipeline.

Disclaimer: docs are still work-in-progress!

Each step provides a defined set of actions. When a pipeline is execute first all pre actions of all Steps are executed, then run actions and so on. Steps can provide labels, so you can easily disable/enable a subset of steps.

A ready to use python app template. Based on pipenv.

How to use the library

BuildStep

The most important basic class is BuildStep. The class is abstract and you have to inherit from it to provide any actual functionality. The most important methods and properties of this class are:

  • Each BuildStep provides a set of step names it is associated with in the steps_provided property. These steps are used for filtering with --steps/--skip-steps command line options.
  • initialize_config provides additional config options a specific class delivered from BuildStep wants to provide.
  • pre_run is optional and should be used for validation and assertions. pre_runs of all BuildSteps are executed before any run method is executed. Its purpose is to allow the abs to quit with error even before any actual build or tests are done. The method can't be blocking and should run fast. If pre_step of any BuildStep fails, run methods of all BuildSteps are skipped.
  • run is the method where actual long-running actions of the BuildStep are executed.
  • cleanup is an optional method used to clean up resources that might have been needed by run but can't be cleaned up until all runs have executed. cleanups are called after any run failed or all of them are done.

BuildStepsFilteringPipeline

BuildStep class provides the steps_provided property, but is not in control of whether it should be executed or not and when. BuildSteps have to be assembled into pipelines. The basic pipeline in BuildStepsFilteringPipeline, which allows you to make a sequential pipeline out of your steps and filter and skip them according to steps_provided they return and command line options --steps/--skip-steps. Each major part of abs execution is combined into a pipeline, like HelmBuildFilteringPipeline used to execute build pipeline with Helm 3 or PytestTestFilteringPipeline which is used to execute tests using pytest once the build pipeline is done.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

step_exec_lib-0.3.1.tar.gz (13.7 kB view details)

Uploaded Source

Built Distribution

step_exec_lib-0.3.1-py3-none-any.whl (15.4 kB view details)

Uploaded Python 3

File details

Details for the file step_exec_lib-0.3.1.tar.gz.

File metadata

  • Download URL: step_exec_lib-0.3.1.tar.gz
  • Upload date:
  • Size: 13.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for step_exec_lib-0.3.1.tar.gz
Algorithm Hash digest
SHA256 645eb96eb8501c8d8c5e491883fc18994d845d423d2b24bf74c681233a671c17
MD5 8b59f5bc83001c30cde623c6df19b817
BLAKE2b-256 c52e02382863e0b59457c3126800c262dd8b06f25b821eba1fb6c5e132280093

See more details on using hashes here.

File details

Details for the file step_exec_lib-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: step_exec_lib-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 15.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for step_exec_lib-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 de4e3001d1fc84e9ac312c026c849817a19722e00c93b994ccce48911f254d76
MD5 fc85ecbe72cc2a8c981d89d5ece36cae
BLAKE2b-256 3c221571c694d1dedcccc41759e18fa8ef17ff2857f2f63d3f92544f882d5205

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page