Skip to main content

A simple Linux command-line utility which submits a job to one of the multiple servers

Project description

ΣΣJob

PyPI version License

ΣΣJob or SumsJob (Simple Utility for Multiple-Servers Job Submission) is a simple Linux command-line utility which submits a job to one of the multiple servers each with limited resources. It will first look for servers with available resources, such as GPUs, and then run the job in that server interactively just as the job is running in your local machine.

Tutorial: Slides

Features

  • Simple to use: one single submit command is all your need
  • Automatically choose available GPUs among all the servers
  • Display the output of the job in real time
  • Kill the job by Ctrl-C
  • Save the output in a log file
  • Transfer back the files you specified

Usage

$ gpuresource

Show the status of GPUs on all servers.

$ submit jobfile jobname

  • jobfile : File to be run
  • jobname : Job name, and also the folder name of the job. If not provided, a random number will be used.

Options:

  • -h, --help : Show this help message and exit
  • -s SERVER, --server SERVER : Server host name
  • --gpuid GPUID : GPU ID to be used; -1 to use CPU only

Installation

Install gpustat in each server.

Then, you can install SumsJob with pip:

$ pip install sumsjob

You need to have a configuration file at ~/.sumsjob/config.py. Use config.py as a template, and modify the values to your configurations.

  • Make sure you can ssh to each server.
  • Run gpuresource to check if everything works. Make sure ~/.local/bin is in your $PATH.

License

GNU GPLv3

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

SumsJob-0.1.1.tar.gz (17.6 kB view hashes)

Uploaded Source

Built Distribution

SumsJob-0.1.1-py3-none-any.whl (18.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page