Skip to main content

Wrap your scripts around a cli.

Project description

pyrunner

A simple Python task runner. Use it as a CLI, or wrap your own scripts with the python interface.

Features

  • Prevent a task to be running twice simultaneously.
  • Prevent a task to be launched again when it is already completed.

It implements this by writing token files .done and .running in a specified task directory. One directory = One task.

Example use cases

Crash recovery

Say you launched multiple tasks for all subdirectories: ls | xargs -I % command %, and the command crashes in the middle. Now how do you run only the unfinished tasks ?

You can use pyrunner to run the your tasks, and then the second run will only start unfinished tasks.

Parallel

It can also be used as a poor man's parallel library. Instead of ls | xargs -P 2 -I % command % you can run ls | xargs -I % pyrunner %/.tokens command % in two shells. Each shell will pick up the pending tasks.

Installation

pip install pyrunner

CLI usage

You can use it as a cli, like this :

pyrunner <tokens-folder> <command>

For example

pyrunner .tokens/ touch file.txt

Will create a file called file.txt.

Another example :

pyrunner .tokens/ bash -c "date > date.txt"

This command will save the current date in the file date.txt. If you run it again it will not run.

Python Usage

You can also wrap your script in a python class. The advantage is that you can define your own arguments, and use them to define the tokens folder. (and avoid duplicating it as an argument like with the pyrunner cli).

For this, you need to wrap your script in a python class.

A basic task that creates a file

from pyrunner import Task

class TouchTask(Task):

    def command(self):
        return "touch"

    def experiment_folder(self, args):
        return "." # return current folder

if __name__ == "__main__":
    TouchTask().run()

More complex examples are in the examples/ directory.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyrunner-0.1.4.tar.gz (4.2 kB view details)

Uploaded Source

Built Distribution

pyrunner-0.1.4-py2.py3-none-any.whl (3.5 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file pyrunner-0.1.4.tar.gz.

File metadata

  • Download URL: pyrunner-0.1.4.tar.gz
  • Upload date:
  • Size: 4.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for pyrunner-0.1.4.tar.gz
Algorithm Hash digest
SHA256 2f00ab841465bb394c5a2eed8e16b4d2075ea5aabfd6143aaafa6802690541c4
MD5 cbf4fe51a667003224a2842df8a375e6
BLAKE2b-256 817bdd7f13c4bc3dc3720f048c6afdd9459ebaf8502cee9ca7a13687e986f80f

See more details on using hashes here.

File details

Details for the file pyrunner-0.1.4-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for pyrunner-0.1.4-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 5de1c5552767c5e240cc93e67b22855b8b368dfeca724d0f472f80cfa198face
MD5 b292552b142e3e65879bff19c9053c87
BLAKE2b-256 bcaa9f2f98ef1afc994341cb216b29ee3c406be054b76a883efaeadc9ed9bf63

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page