A CLI tool to manage script queue
Project description
Script Queue Manager
A small suite of simple CLI tools to queue up scripts to be executed. A simple use case is that of having to repeatedly execute the same scripts with different parameters and each script takes hours to execute. (eg - I personally use it to queue my deep learning training tasks)
Installing from pip
pip install -i https://test.pypi.org/simple/ script-queue-manager
Getting Started
I highly recommend that you fork the project by clicking on the Fork
button and then cloning your fork using the command mentioned below.
git clone <your fork URL>
Prerequisites
- Python 3.5+
- A code editor or IDE(preferable PyCharm)
Setting up for Development
Follow the steps mentioned below to get your dev instance running.
-
After cloning your fork, open the project in an IDE or code editor of you choice.
-
It is highly advisable that you create a new Python environment for the project. Follow the instructions given here.
-
Navigate to the project directory and install the
requirement.txt
packages.cd <project-directory> pip install -r requirements.txt
-
Build the project:
python setup.py install
-
Use the
pusher
to push a script to the task queue.sqm-pusher -s "cat x.txt"
-
Start the
executor
tool.sqm-executor
If you want to start it as a background process, in Ubuntu run
sqm-executor &
Execution Model
The executor fetches an object from the front of the queue and prepares a script as PRE_SCRIPT + QUEUE_SCRIPT
. Here, PRE_SCRIPT
can be given as input to the executor using the flag -p
as shown below:
sqm-executor -p "path-to-the-file-containing-the-prescripts"
The sleeping interval for the executor to sleep in between tasks can be provided using the flag -s
. By default it is 15s.
The executor produces logs for every task it executes. The file path for this log file can be provided using the -l
flag. By default it is located at $HOME/sqm/logs.txt
.
Tests
Not yet implemented
Built With
- Persistqueue - File based queue.
Contributing
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
Authors
- Mayank Vaidya - Initial work
See also the list of contributors who participated in this project.
License
This project is licensed under the MIT License - see the LICENSE.md file for details
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for script_queue_manager-0.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 876a9bddb3fcfb72757677ac6d5a9d1a2fd9f0a77d7d9b76f259b6b13e477806 |
|
MD5 | d097dbc567a89ac4921c52e6680d8e7a |
|
BLAKE2b-256 | af6ceff3f42d50f92ff5be77b369503def6f9999359a2a4b068d415df6047ba1 |
Hashes for script_queue_manager-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f1c84c1710a65d7359032c1b36cf60eb609c98807ac13c71366bff72e07446ae |
|
MD5 | 6558e3fd94b8fbd3db626bcd42f6bbe5 |
|
BLAKE2b-256 | 8bf9cc3ef74736590a1d2f825b8ec54f83e1975466b4a678553d546b3eb47a55 |