This release is a pre-release and may not be stable for production use.
Toil is a massively scalable pipeline management system, written entirely in Python. Toil runs as easily on a laptop as it does on a bare-metal cluster or in the cloud, thanks to support for many batch systems, including Grid Engine, Parasol, and a custom Mesos framework.
Toil is robust, and designed to run in highly unreliable computing environments like Amazon’s Spot Market. Towards this goal, Toil does not rely on a distributed file system. Instead, Toil abstracts a pipeline’s global storage as a JobStore that can be stored either locally or on AWS. The result of this abstraction is a robust system that can be resumed even after an unexpected shutdown of every node in the cluster that resulted in the loss of all local data.
Writing a Toil script requires only a knowledge of basic Python, with Toil “Jobs” as the elemental unit of work in a Toil workflow. A Job can dynamically spawn other Jobs as needed, leading to an intuitive and powerful control over the pipeline.
Prerequisites
Python 2.5 < 3.0
pip 7.x
Apache Mesos 0.22.1, if using the Mesos batch system. This is Brew installable on OSX via:
brew install mesos
Git, if cloning from the Toil Github Repository
Installation
Toil uses setuptool’s extras syntax for dependencies of optional features, like the Mesos batch system and the AWS JobStore. To install Toil with these extras, specify the features you would like to include when pip installing:
pip install toil[aws,mesos]
Building & Testing
This is only required if cloning from Git. Running:
make develop
will install Toil in editable mode. You can also specify extras to use in develop mode as follows:
make develop extras=[mesos,aws]
To run the tests, cd into the toil root directory and run:
make test
Finally, running:
make
by itself will print help for testing and building.
Release files for toil 3.0.6a1.dev377
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| toil-3.0.6a1.dev377.tar.gz | 108.4 kB | Without provenance | Details for toil-3.0.6a1.dev377.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| toil-3.0.6a1.dev377-py2.7.egg | Legacy Egg format | - | - | Details for toil-3.0.6a1.dev377-py2.7.egg |
Total release size: 422.9 kB
Release files / toil-3.0.6a1.dev377.tar.gz
| Download URL | toil-3.0.6a1.dev377.tar.gz |
|---|---|
| Size | 108.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
474089bf044d54943bb84a5bf864af55c5a1af3e4745b86625a942be18ef4321
|
|
BLAKE2b-256 checksum How to use checksums |
02f3e7aaf455e69bda3bbc193bf1340aa3314f4561b9cefc8e6219a6eba5c7d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / toil-3.0.6a1.dev377-py2.7.egg
| Download URL | toil-3.0.6a1.dev377-py2.7.egg |
|---|---|
| Size | 314.5 kB |
| Tags | Egg |
|
SHA-256 checksum How to use checksums |
29c7e51aebc6bca3269c22cf1968494350b177e2f0f4c4eeee8d0774f7021f61
|
|
BLAKE2b-256 checksum How to use checksums |
6c810254cc67c0e33ef5c9997988990aec5c76f1990595649b773790540c5bd0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for toil 3.0.6a1.dev377
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
