Skip to main content

database backed taskfarm controller

Project description

taskfarm

This package solves the problem of managing a loosely coupled taskfarm where there are many tasks and the workers are entitrly independent of each other. Instead of using a farmer process a database is used to hand out new tasks to the workers. The workers contact a web application via http(s) to get a new task.

You can use the taskfarm-worker to connect to the taskfarm service.

Setup

After installing the python package you need to connect to a database. For testing purposes you can use sqlite. However, sqlite does not allow row locking so if you use parallel workers a task may get assigned to the multiple workers.

You can set the environment variable DATABASE_URL to configure the data base connection. For example

export DATABASE_URL=sqlite:///app.db

or

export DATABASE_URL=postgresql://user:pw@host/db

You then need to create the tables by running

adminTF --init-db

You can then create some users

adminTF -u some_user -p some_password

These users are used by the worker to connect to the service.

test package taskfarm Documentation Status

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

taskfarm-0.3.0.tar.gz (26.5 kB view details)

Uploaded Source

Built Distribution

taskfarm-0.3.0-py3-none-any.whl (20.9 kB view details)

Uploaded Python 3

File details

Details for the file taskfarm-0.3.0.tar.gz.

File metadata

  • Download URL: taskfarm-0.3.0.tar.gz
  • Upload date:
  • Size: 26.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.8.0 tqdm/4.30.0 CPython/3.8.10

File hashes

Hashes for taskfarm-0.3.0.tar.gz
Algorithm Hash digest
SHA256 fe21e72a47317775f050ea9d0952e321f0940f700e5cb54f47576e8d41df9969
MD5 031a3e932c4b4addb721581c58981366
BLAKE2b-256 2fe3cfc5532a86f543cfde7f0827e47a05382d06274151ff1c20fc46852b11ac

See more details on using hashes here.

File details

Details for the file taskfarm-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: taskfarm-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 20.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.8.0 tqdm/4.30.0 CPython/3.8.10

File hashes

Hashes for taskfarm-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e610a3025b2cc8e8b45e13cd8b0fd7c2e01fd2c36a17fbbfeeea20c8eee9e3a9
MD5 e2fa0ed77857d5c42fe2572f14da8ca5
BLAKE2b-256 fe96d3fe371b1fb5381d4e9772218731e33fb98ee056102d6ec53eeb11ac7f0e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page