Dequindre /_de-KWIN-der_/ (n.): A minimalist scheduler.
Project description
Dequindre /_de-KWIN-der_/ (n.): A minimalist scheduler.
Vision: Simplify Workflow Automation
dequindre
aims to simplify workflow automation. It is part of a larger vision to teach the fundamentals and best practices to practicing and aspiring data scientists and data engineers.
What is dequindre?
dequindre
is a minimalist scheduler you can use to:
- quickly configure python workflows at home or at work,
- run dependent tasks in separate python environments, and
- learn the fundamentals and best practices of scheduling workflows.
Features
- Automated workflow scheduling
- Run your Python tasks in any pre-defined environments
dequindre
facilitates virtualenv, conda, and pipenv environments
- Supports dynamic workflow configuration also seen in Airflow
- Pure Python: Relies entirely from Python built-ins.
- Cross-Python compatible: Supports Python 2 and Python 3
- Cross-platform: Windows and Unix style environments
- Documented examples and configuration
Basic Example
First, install dequindre
with pip install dequindre
. Then, in the
REPL or in a schedule.py
file,
>>> from dequindre import Task, DAG, Dequindre
>>>
>>> ## define tasks and environments
>>> pour_water = Task('pour_water.py)
>>> boil_water = Task('boil_water.py')
>>> prep_infuser = Task('prep_infuser.py)
>>> steep_tea = Task('steep_tea.py')
>>>
>>> ## define runtime dependencies
>>> make_tea = DAG(dependencies={
... boil_water: {pour_water},
... steep_tea: {boil_water, prep_infuser}
... })
>>>
>>> ## run tasks
>>> dq = Dequindre(make_tea, validate_conda=False)
>>> dq.get_schedules()
defaultdict(<class 'set'>, {
1: {Task(prep_infuser.py), Task(pour_water.py)},
2: {Task(boil_water.py)},
3: {Task(steep_tea.py)}})
Getting Started
Dequindre is has two requirements: conda and python.
Installing
$ pip install dequindre
Contributing
If you're interested in contributing to Dequindre, raise an issue, make a pull request to dev
, and reach out to the author, vogt4nick.
Please read contributing.md for details on our code of conduct, and the process for submitting pull requests to us.
Versioning
We use SemVer for versioning. For the versions available, see the tags on this repository.
License
This project is licensed under the MIT License - see the license.md file for details.
Acknowledgements
Thank you, Dynatrace, for facilitating the early development of Dequindre during Innovation Day, February 2019.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dequindre-0.7.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 953947430d4ca20e8338d5da1a0e60bec6d45c1e21f83d4d2697bc6f169fa4d0 |
|
MD5 | ea502ba51093db8470162ff68236ea25 |
|
BLAKE2b-256 | 7f72d5f2999dc4329866ba76c5dee8da24a074b786bb51b14e51ac998778f8bd |