pytask is a workflow management system that facilitates reproducible data analyses. Its features include:
- Automatic discovery of tasks.
- Lazy evaluation. If a task, its dependencies, and its products have not changed, do not execute it.
- Debug mode. Jump into the debugger if a task fails, get feedback quickly, and be more productive.
- Repeat a task with different inputs. Loop over task functions to run the same task with different inputs.
- Select tasks via expressions. Run only a subset of tasks with expressions and marker expressions.
- Easily extensible with plugins. pytask is built on top of pluggy, a plugin management framework, which allows you to adjust pytask to your needs. Plugins are available for parallelization, LaTeX, R, and Stata and more can be found here. Learn more about plugins in this tutorial.
Installation
pytask is available on PyPI and on Anaconda.org. Install the package with
$ pip install pytask
or
$ conda install -c conda-forge pytask
Color support is automatically available on non-Windows platforms. On Windows, please, use Windows Terminal, which can be, for example, installed via the Microsoft Store.
To quickly set up a new project, use the cookiecutter-pytask-project template or start from other templates or example projects.
Usage
A task is a function that is detected if the module and the function name are prefixed
with task_. Here is an example.
# Content of task_hello.py.
from pathlib import Path
from pytask import Product
from typing_extensions import Annotated
def task_hello_earth(path: Annotated[Path, Product] = Path("hello_earth.txt")):
path.write_text("Hello, earth!")
-
The purpose of the task is to create the file
hello_earth.txtand add some content. -
To tell pytask that
hello_earth.txtis a product and not an input, use theProductannotation.(If you are not used to type annotations, do not worry. pytask also offers simpler interfaces without type annotations.)
-
Since you pass a
pathlib.Pathto the function, pytask will check whether the file exists after the function is executed.
To execute the task, enter pytask on the command-line
Documentation
You find the documentation https://pytask-dev.readthedocs.io/en/stable with tutorials and guides for best practices.
Changes
Consult the release notes to find out about what is new.
License
pytask is distributed under the terms of the MIT license.
Acknowledgment
The license also includes a copyright and permission notice from pytest since some modules, classes, and functions are copied from pytest. Not to mention how pytest has inspired the development of pytask in general. Without the excellent work of Holger Krekel and pytest's many contributors, this project would not have been possible. Thank you!
pytask owes its beautiful appearance on the command line to rich, written by Will McGugan.
Repeating tasks in loops is inspired by ward written by Darren Burns.
Citation
If you rely on pytask to manage your research project, please cite it with the following key to help others to discover the tool.
@Unpublished{Raabe2020,
Title = {A Python tool for managing scientific workflows.},
Author = {Tobias Raabe},
Year = {2020},
Url = {https://github.com/pytask-dev/pytask}
}
1 maintainer
Release files for pytask 0.4.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pytask-0.4.2.tar.gz | 103.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pytask-0.4.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 223.7 kB
Release files / pytask-0.4.2.tar.gz
| Download URL | pytask-0.4.2.tar.gz |
|---|---|
| Size | 103.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
32c3af8792436575d53e6a2d05369e0a65fc580de817800d76aed3fba97796b2
|
|
BLAKE2b-256 checksum How to use checksums |
e873e435fa58d8dc1b8a4d6c9095be6654ad0a71e176fd6c017ceed0f47ae6a8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.6
|
Release files / pytask-0.4.2-py3-none-any.whl
| Download URL | pytask-0.4.2-py3-none-any.whl |
|---|---|
| Size | 120.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f86a40622bb0b2a1dad54bdee312f64669cb30548b169a50b5d2c310b0980ae7
|
|
BLAKE2b-256 checksum How to use checksums |
7582c908b38f8be8830e8567f427cc8dcb6d096f3512d9b059b3e59e11e49fa6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.6
|
