SumsJob 0.1.1

A simple Linux command-line utility which submits a job to one of the multiple servers

ΣΣ_Job

ΣΣ_Job or Sums_Job (Simple Utility for Multiple-Servers Job Submission) is a simple Linux command-line utility which submits a job to one of the multiple servers each with limited resources. It will first look for servers with available resources, such as GPUs, and then run the job in that server interactively just as the job is running in your local machine.

Tutorial: Slides

Features

• Simple to use: one single submit command is all your need
• Automatically choose available GPUs among all the servers
• Display the output of the job in real time
• Kill the job by Ctrl-C
• Save the output in a log file
• Transfer back the files you specified

Usage

$ gpuresource

Show the status of GPUs on all servers.

$ submit jobfile jobname

• jobfile : File to be run
• jobname : Job name, and also the folder name of the job. If not provided, a random number will be used.

Options:

• -h, --help : Show this help message and exit
• -s SERVER, --server SERVER : Server host name
• --gpuid GPUID : GPU ID to be used; -1 to use CPU only

Installation

Install gpustat in each server.

Then, you can install Sums_Job with pip:

$ pip install sumsjob

You need to have a configuration file at ~/.sumsjob/config.py. Use config.py as a template, and modify the values to your configurations.

• Make sure you can ssh to each server.
• Run gpuresource to check if everything works. Make sure ~/.local/bin is in your $PATH.

License

GNU GPLv3