database backed taskfarm controller
Project description
taskfarm
This package solves the problem of managing a loosely coupled taskfarm where there are many tasks and the workers are entitrly independent of each other. Instead of using a farmer process a database is used to hand out new tasks to the workers. The workers contact a web application via http(s) to get a new task.
You can use the taskfarm-worker to connect to the taskfarm service.
Setup
After installing the python package you need to connect to a database. For testing purposes you can use sqlite. However, sqlite does not allow row locking so if you use parallel workers a task may get assigned to the multiple workers.
You can set the environment variable DATABASE_URL to configure the data base connection. For example
export DATABASE_URL=sqlite:///app.db
or
export DATABASE_URL=postgresql://user:pw@host/db
You then need to create the tables by running
adminTF --init-db
You can then create some users
adminTF -u some_user -p some_password
These users are used by the worker to connect to the service.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file taskfarm-0.3.0.tar.gz
.
File metadata
- Download URL: taskfarm-0.3.0.tar.gz
- Upload date:
- Size: 26.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.8.0 tqdm/4.30.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe21e72a47317775f050ea9d0952e321f0940f700e5cb54f47576e8d41df9969 |
|
MD5 | 031a3e932c4b4addb721581c58981366 |
|
BLAKE2b-256 | 2fe3cfc5532a86f543cfde7f0827e47a05382d06274151ff1c20fc46852b11ac |
File details
Details for the file taskfarm-0.3.0-py3-none-any.whl
.
File metadata
- Download URL: taskfarm-0.3.0-py3-none-any.whl
- Upload date:
- Size: 20.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.8.0 tqdm/4.30.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e610a3025b2cc8e8b45e13cd8b0fd7c2e01fd2c36a17fbbfeeea20c8eee9e3a9 |
|
MD5 | e2fa0ed77857d5c42fe2572f14da8ca5 |
|
BLAKE2b-256 | fe96d3fe371b1fb5381d4e9772218731e33fb98ee056102d6ec53eeb11ac7f0e |