Task handlinlg.
Project description
TaskSquirrel
TaskSquirrel executes tasks from a Tasqfile.yml
. It is similar to make
or invoke
. It also supports Kubernetes workflows like Argo
and Airflow
.
Differences from other tools are:
- configuration is in YAML
- it supports multiple scripts / languages
- it supports Jinja2 templating
- it has built-in support for executing code in Docker and on Kubernetes clusters
- it is completely serverless: all it requires for interacting with a K8s cluster is Kubectl
TaskSquirrel is a good replacement for invoke
and make
even if you don't use
the container-related features, but it is particularly well suited to data analysis
and deep learning workflows.
Commands
tasq --help
-- display helptasq list
-- list all commandstasq run command
-- run a command
Example
Simple replacement for invoke
tasks:
venv:
- doc: build the virtual environment
- script: |
test -d venv || virtualenv venv
venv/bin/pip3 install -U -r requirements.txt
newversion:
- doc: increase the patch version number
- python: |
version = map(int, open("VERSION").read().strip().split("."))
version[-1] += 1
with open("VERSION", "w") as stream:
stream.write(".".join(map(str, version)))
publish:
- doc: publish to PyPI
- script: twine upload
Kubernetes Example
params:
cluster: "mycluster"
image: "mycompute"
tasks:
image:
docker_container: {{ image }}
dockerfile: ./Docker
push: tmbdev/{{ image }}
context:
- simple.key
- scripts/start.sh
kutest:
- doc: start a job on K8s and wait for its completion
- task: image
- with_braceexpand: "{000..999}"
param: shard
tasks:
- kubectl_apply: hello
template_name: myjob
params:
id: hello-{{ shard }}
script: |
echo hello-{{ shard }};
sleep 30
- kubectl_wait:
kind: jobs
prefix: hello-
- script: kubectl delete jobs --all
k8s_templates:
myjob: |
apiVersion: batch/v1
kind: Job
metadata:
name: "{{ id }}"
spec:
template:
spec:
containers:
- name: main
image: "{{ image }}"
command: ["/bin/bash", "-c", "{{ script }}"]
restartPolicy: Never
backoffLimit: 4
Status
It's pre-alpha.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tasq-0.0.3.tar.gz
(4.7 kB
view details)
Built Distribution
tasq-0.0.3-py3-none-any.whl
(4.9 kB
view details)
File details
Details for the file tasq-0.0.3.tar.gz
.
File metadata
- Download URL: tasq-0.0.3.tar.gz
- Upload date:
- Size: 4.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
c29204789658808ef9d0696eb274da597a3829e68f80853a9ac21e6675cd996e
|
|
MD5 |
c6b7313d7fb571de904ab801f0050d16
|
|
BLAKE2b-256 |
dd2e9c3d657d4d7504f48abdf1407f95143701465db691fdaad77caad1b76581
|
File details
Details for the file tasq-0.0.3-py3-none-any.whl
.
File metadata
- Download URL: tasq-0.0.3-py3-none-any.whl
- Upload date:
- Size: 4.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
8d7d36c22b14c572bbaef56151928a211fdac34571d4c7cc925bf8cf602b7900
|
|
MD5 |
5486e53b92284137a2bb6918f4e0ff72
|
|
BLAKE2b-256 |
52d35aa0d59481ef2026be6fcedce9fc1d468a8d65a17a0a83d54eb25baf538e
|