A Python 3 tool to harness the google cloud to run shell scripts
Project description
Turbine
A Python 3 tool to harness the google cloud to run shell scripts. In particular, if you have
- a docker image, and
- many independent, idempotent tasks for that image written as shell scripts,
Turbine allows you to run those tasks on many VMs in parallel on GCE. Turbine is a lot like
- Google Cloud Run except you don't need to hit an HTTP endpoint and it supports large machine types and accelerators;
- Google Cloud Functions except it supports large machine types and accelerators; and
- Batch on GKE except I understand it, it is significantly lighter-weight than Kubernetes, and it provides a path for TPU access.
Installation
For most users, the recommended method to install is via pip:
pip install turbine
Dependencies:
- google-cloud-pubsub
- google-cloud-storage
- google-cloud-logging
- google-api-python-client
See requirements.txt for detailed version information, if needed.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
turbine-1.0.0.tar.gz
(10.2 kB
view hashes)