stjudecloud-oliver 1.0.7

An opinionated Cromwell orchestration system

Oliver

An opinionated Cromwell orchestration manager.

Getting Started

Conda

Oliver is distributed as a package using the community-curated Anaconda repository, conda-forge. We recommend that you first follow the instructions included in the conda-forge documentation to get everything set up!

conda install oliver -c conda-forge

Python Package Index

You can also install Oliver using the Python Package Index (PyPI).

pip install stjudecloud-oliver

Please refer to the guides in the docs/ folder for more information.

Guide Name	Link
Advanced Usage	Link
Configuration	Link
Submitting Jobs	Link

Usage

oliver -h

The following subcommands are currently supported.

Subcommand	Short Command	Description
aws		All subcommands related to Cromwell on AWS.
azure		All subcommands related to Cromwell on Azure.
aggregate	a	Aggregate all results to a local or cloud folder for a run.
batches	b	Explore batches of jobs submitted to Cromwell.
configure		Configure Oliver with default options.
config		Set or get a single config value from Oliver.
inputs		Find all reported outputs for a given workflow.
inspect	i	Describe the state of a Cromwell workflow.
kill	k	Kill a workflow running on a Cromwell server.
logs	l	Find all reported logs for a given workflow.
outputs	o	Find all reported outputs for a given workflow.
retry	re	Resubmit a workflow with the same parameters.
runtime	ru	Get the runtime attributes used for a specific call.
status	st	Report various statistics about a running Cromwell server.
submit	su	Submit a workflow to the Cromwell server.

Development

If you are interested in contributing to the code, please first review our CONTRIBUTING.md document. To bootstrap a development environment, please use the following commands.

# Clone the repository
git clone [e-mail]:stjudecloud/oliver.git
cd oliver

# Link the package with your current Python environment
python setup.py develop

# Ensure pre-commit is installed to automatically format
# code using `black`.
brew install pre-commit
pre-commit install
pre-commit install --hook-type commit-msg

Tests

Oliver provides a (currently patchy) set of tests — both unit and end-to-end. To get started with testing, you'll need to bootstrap a Docker test environment (one-time operation).

# Start development environment
docker image build --tag oliver .
docker-compose up --build  -d

alias docker-run-oliver="docker container run \
  -it \
  --rm \
  --network oliver_default \
  --mount type=bind,source=$PWD/seeds,target=/opt/oliver/seeds \
  --mount type=bind,source=$PWD/oliver,target=/opt/oliver/oliver \
  --mount type=bind,source=$PWD/scripts,target=/opt/oliver/scripts \
  --mount type=bind,source=$PWD/tests,target=/opt/oliver/tests \
  oliver"

# Seed development environment (make sure Cromwell is live first!)
docker-run-oliver bash seeds/seed.sh http://cromwell:8000 seeds/wdl/hello.wdl
docker-run-oliver pytest

To reset your entire docker-compose environment, you can run the following:

docker-compose down

docker image rm oliver:latest
docker image rm oliver_cromwell:latest
docker image rm mysql:5.7
docker volume rm oliver_mysql_data
docker network rm oliver_default

docker image build --tag oliver .
docker-compose up --build -d

Author

👤 St. Jude Cloud Team

• Website: https://stjude.cloud
• Twitter: @StJudeResearch
• Github: @stjudecloud

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page. You can also take a look at the contributing guide.

📝 License

Copyright © 2020 St. Jude Cloud Team.
This project is MIT licensed.