CWL runner for Kubernetes
Project description
calrissian
CWL on Kubernetes
Overview
Calrissian is a CWL implementation designed to run individual steps as Pods in a kubernetes cluster
It is in development and includes a simple workflow (revsort single / array) to reverse and sort the contents of text files.
Preparing Openshift
The openshift/
directory contains YAML files that demonstrate the basic functionality. To configure Openshift, do the following:
-
Create a project in openshift (e.g.
calrissian-demo-project
). Note that namespaces must be unique, so ifcalrissian-demo-project
is already in use on the cluster, you will need to choose a different name.oc new-project calrissian-demo-project
-
Create a role in your project to allow managing
Pod
s:oc create role pod-manager-role --verb=create,delete,list,watch --resource=pods
-
Bind your project's default service account access to these roles. Calrissian running inside the cluster will use this service account to make Kubernetes API calls.
oc create rolebinding pod-manager-default-binding --role=pod-manager-role --serviceaccount=calrissian-demo-project:default
-
Create the BuildConfig - this allows Openshift to build the source code into a Docker image and starts a build automatically. If you wish to build a different branch or repo, edit this file.
oc create -f openshift/BuildConfig.yaml
-
Create the VolumeClaims - these are the storage locations that will be shared between pods, and must support read-write many access.
oc create -f openshift/VolumeClaims.yaml
Running the example
With the VolumeClaims and Build processes in place, you can run the example with 2 jobs. These jobs use the docker image built by the BuildConfig, so they cannot be run until the build completes.
-
Stage the input data and workflow onto the
input-data
volume with the StageInputDataJob.oc create -f openshift/StageInputDataJob.yaml
-
Run the workflow engine with the CalrissianJob
oc create -f openshift/CalrissianJob-revsort.yaml
-
Job pods can be monitored from the openshift web interface or via command-line
oc logs -f job/calrissian-revsort-array
Notes
- Calrissian will delete completed jobs for individual steps, but you may want to delete the
calrissian-revsort-array
job manually after running it.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file calrissian-0.3.1-py3.6.egg
.
File metadata
- Download URL: calrissian-0.3.1-py3.6.egg
- Upload date:
- Size: 41.8 kB
- Tags: Egg
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
f5998739e6c145f0373704140e2370183ccc82caa14feda25024631b567a1a48
|
|
MD5 |
79d3acb308a56e378c4ab9000c177777
|
|
BLAKE2b-256 |
add94eef5aaed1defa68e3d96df7bf99c89acd467023bf26a387f6997663e778
|
File details
Details for the file calrissian-0.3.1-py2.py3-none-any.whl
.
File metadata
- Download URL: calrissian-0.3.1-py2.py3-none-any.whl
- Upload date:
- Size: 20.0 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
b01ed8793d0ac559c479dfa99fb3c3f58f73d853bc5c819db497d1601c7c1c5d
|
|
MD5 |
6bccd22a6ea16c9feb57628031fa85b7
|
|
BLAKE2b-256 |
811a414893370e16fbbf4ebf34297affff9f7434f928689dbdde7b67bc1b28e7
|