Otter — Open Targets' Task ExecutoR
Otter is the task execution framework used in the Open Targets data Pipeline.
It provides an easy to use API to implement generic tasks that are then used by describing the flow in a YAML configuration file.
Take a look at a Simple example.
Features
This is a list of what you get for free by using Otter:
- Parallel execution: Tasks are run in parallel, and Otter will take care of the dependency planning.
- Async supported: Tasks can be either sync or async.
- Declarative configuration: Steps are described in a YAML file, as list of tasks with different specifications. The task themselves are implemented in Python enabling a lot of flexibility.
- Logging: Otter uses the loguru library for logging. It handles all the logging related the task flow, and also logs into the manifest (see next item).
- Manifest: Otter manages a manifest file that describes a pipeline run. It is used to both for debugging and for tracking the provenance of the data. A series of simple JQ queries can be used to extract information from it (see Useful JQ queries).
- Error management: Otter will stop the execution of the pipeline if a task fails, and will log the error in the manifest.
- Scratchpad: A place to store variables that can be overwritten into the configuration file (something like a very simple templating engine), enabling easy parametrization of runs, and passing of data between tasks.
- Utilities: Otter provides interfaces to use Google Cloud Storage and other remote storage services, and a bunch of utilities to help you write tasks.
Documentation
See it in here.
Development
[!IMPORTANT] Remember to run
make devbefore starting development. This will set up a very simple git hook that does a few checks before committing.
1 maintainer
Metadata
1 maintainer
Release files for opentargets-otter 26.9.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| opentargets_otter-26.9.1.tar.gz | 233.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| opentargets_otter-26.9.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 305.6 kB
Release files / opentargets_otter-26.9.1.tar.gz
| Download URL | opentargets_otter-26.9.1.tar.gz |
|---|---|
| Size | 233.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b4cb005c6a88a4c00d51b174dfdfc77228162b2693e185a542707da753faecf5
|
|
BLAKE2b-256 checksum How to use checksums |
30e86176c92420f3bf62b431590452b7916d03a3c61c369a3a9e243193a6c1bf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 26, 2026.
Transparency logRelease files / opentargets_otter-26.9.1-py3-none-any.whl
| Download URL | opentargets_otter-26.9.1-py3-none-any.whl |
|---|---|
| Size | 72.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
54af2d7eb569abfdfa21fece69d01e6fe547f674a5954dd1d15d76a8c3d3d768
|
|
BLAKE2b-256 checksum How to use checksums |
4121acbfdbc95a11d7a31ad045c21e59b39c118fda403bfb06a2a97d933d1ef2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 26, 2026.
Transparency log
