Skip to main content

Pipeline orchestration and execution

Project description

Conducto

Conducto is a CI/CD tool

Here are some things that set it apart:

  • Use Python instead of YAML (Javascript API coming soon).
  • Explore a tree (not a DAG).
  • Fix with local editors, debug in local containers.
  • Move workloads between --local and --cloud with a single argument.

Computational Pipelines

Consider a computational task that breaks into pieces (and maybe its pieces have pieces too).

Conducto organizes those pieces into trees that we call pipelines. At conducto.com we host pages that let you view and control them.

Pipeline Screenshot

Centralize Insights, Work Wherever

Conducto is good for those long running tasks that people like to check in on. It provides a place where they can watch them happen (even if they're running on your dev box).

Conducto pipelines are fully encapsulated in containers, so they're portable. All you need is a Docker, a web browser, and a Conducto account (which is free).

The only paid feature is cloud mode, which is where we provide the compute power instead of leaning on your local Docker daemon.

Try It

You'll need Python 3.6 or higher and Docker. Use pip to install:

pip install conducto

A Sample Pipeline

Start with a script like this:

# pipeline.py
import conducto as co

py_code = 'print("Hello")'
js_code = 'console.log("World!")'

def pipeline() -> co.Serial:
    root = co.Serial()
    root["hello from python"] = co.Exec(f"python -c '{py_code}'")
    root["world from javascript"] = co.Exec(f"echo '{js_code}' | node -",
                                            image="node:current-alpine")
    return root

if __name__ == "__main__":
    co.main(default=pipeline)

Launch it like this:

python pipeline.py --local

You'll be prompted to create a Conducto account, and then you'll see your pipeline.

Contact

If you want to chat, join us on Slack.

If you run into problems feel free to use the conducto tag on Stack Overflow or file a GitHub issue on this repo.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

conducto-0.1.19-py3-none-any.whl (178.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page