An opinionated Cromwell orchestration system
- Workflow Orchestration. Easily submit, inspect, kill, and retry workflows in a Cromwell environment.
- Better Job Tracking. Jobs can be associated with names and job groups to enable better status reporting.
- Dynamic Argument Parsing. Specify inputs and options on the command line rather than editing JSON files.
- Third-party Cloud Integrations. Use the
azuresubcommands to explore cloud-specific functionality.
📚 Getting Started
Oliver is distributed as a package using the community-curated Anaconda repository, conda-forge. You'll need to install conda, and we recommend that you first follow the instructions included in the conda-forge documentation to get everything set up!
conda install oliver -c conda-forge
Python Package Index
You can also install Oliver using the Python Package Index (PyPI).
pip install stjudecloud-oliver
Next, we recommend that you configure oliver so that common arguments can be saved. By default, Oliver will prompt you for the answers interactively.
If you are setting up Oliver programmatically, you can accept a default configuration (
oliver configure --defaults) and edit from there using
🚌 A Quick Tour
At its foundation, Oliver is an opinionated job orchestrator for Cromwell. Commonly, you will want to use it to submit a job, inspect a job's status, kill a job, retry a job (possibly with different parameters), and organize job results.
If you're interested in a complete overview of Oliver's capabilities, please see the documentation pages.
Submit a Job
The simplest possible job submission is one which submits a simple workflow with one or more input JSON file(s) and/or key-value pair(s).
oliver submit workflow.wdl inputs.json input_key=input_value
You can similarly set workflow options and labels by prepending arguments with
# works for files too! oliver submit workflow.wdl @option=foo %label=bar
Please see the docs for more details on job submission.
Inspect a Job
Once a job is submitted, you can interrogate the Cromwell server about its status.
oliver inspect workflow-id
If you aren't sure what workflow identifier was given to your job, you can easily track it down using the
# detailed view, which shows individual workflow statuses oliver status -d
Kill a Job
If, for whatever reason, you'd like to stop a job, you can use Oliver to instruct Cromwell to do so.
oliver kill workflow-id
Retry a Job
Retrying a workflow is similarly easy: even if you need to override previously set parameters (e.g. increase the memory capacity for a task).
# override previous inputs by specifying arguments (the same way as you would for `submit`). oliver retry workflow-id
If you are interested in contributing to the code, please first review our CONTRIBUTING.md document. To bootstrap a development environment, please use the following commands.
# Clone the repository git clone firstname.lastname@example.org:stjudecloud/oliver.git cd oliver # Link the package with your current Python environment python setup.py develop # Ensure pre-commit is installed to automatically format # code using `black`. brew install pre-commit pre-commit install pre-commit install --hook-type commit-msg
Oliver provides a (currently patchy) set of tests — both unit and end-to-end. To get started with testing, you'll need to bootstrap a Docker test environment (one-time operation).
# Start development environment docker image build --tag oliver . docker-compose up --build -d alias docker-run-oliver="docker container run \ -it \ --rm \ --network oliver_default \ --mount type=bind,source=$PWD/seeds,target=/opt/oliver/seeds \ --mount type=bind,source=$PWD/oliver,target=/opt/oliver/oliver \ --mount type=bind,source=$PWD/scripts,target=/opt/oliver/scripts \ --mount type=bind,source=$PWD/tests,target=/opt/oliver/tests \ --entrypoint '' \ oliver:latest" # Seed development environment (make sure Cromwell is live first!) docker-run-oliver bash seeds/seed.sh http://cromwell:8000 seeds/wdl/hello.wdl docker-run-oliver pytest --cov=./ --cov-report=xml
To reset your entire docker-compose environment, you can run the following:
docker-compose down docker image rm oliver:latest docker image rm oliver_cromwell:latest docker image rm mysql:5.7 docker volume rm oliver_mysql_data docker network rm oliver_default docker image build --tag oliver . docker-compose up --build -d
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size stjudecloud_oliver-1.1.0-py3-none-any.whl (48.3 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size stjudecloud-oliver-1.1.0.tar.gz (36.8 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for stjudecloud_oliver-1.1.0-py3-none-any.whl