gridengine-goodies: Enhancements to gridengine

The gridengine-goodies package provides a few command line scripts that enhance the usability of the gridengine scheduling system.

qsubx: an improved qsub

qsubx allows to submit multiple jobs at a time while keeping at most N jobs in the queue. This way you can submit tons of jobs and still be polite to your fellow users.

qsubx -n 100 job_*.sh

Note that N=100 is the default. qsubx will not return until all the jobs have been submitted.

qdelx: delete jobs via regular expressions

To delete jobs by name

qdelx sim_T*_rho*

To delete all jobs waiting in the queue

qdelx qw

qstatx: an improved qstat

It provides better layout and full job names (adapted from a code of Jeet Sukumaran). The main benefit of qstatx is to provide full (or almost full) job names and better layout.

------------------------------------------------------------------------------------------------
id      name      owner  submitted            state  queue  node                    slots
------------------------------------------------------------------------------------------------
124905  sleep_10  coslo  2018-10-09 15:54:42  r      all.q  varda.coulomb.univ-mon  1    
124906  sleep_10  coslo  2018-10-09 15:54:42  r      all.q  varda.coulomb.univ-mon  1    
124907  sleep_10  coslo  2018-10-09 15:54:42  r      all.q  varda.coulomb.univ-mon  1    
124908  sleep_10  coslo  2018-10-09 15:54:42  r      all.q  varda.coulomb.univ-mon  1    
124909  sleep_10  coslo  2018-10-09 15:54:44  qw                                    1    
124910  sleep_10  coslo  2018-10-09 15:54:44  qw                                    1    
124911  sleep_10  coslo  2018-10-09 15:54:44  qw                                    1    
124912  sleep_10  coslo  2018-10-09 15:54:44  qw                                    1    
------------------------------------------------------------------------------------------------
Running  :   4 (slots:    4)
Hold     :   0 (slots:    0)
Pending  :   4 (slots:    4)
Deletion :   0 (slots:    0)
Error    :   0 (slots:    0)
------------------------------------------------------------------------------------------------

qexec: execute a command on the cluster

Execute a command on the computing nodes directly. The job file is created on the fly and stored in a temporary directory under the user .local/share folder. The behavior is similar to qsub -b y <command> but qexec is more flexible and provide persistent access to job files and logs if needed.

This job will sleep for 10 seconds

qexec -c sleep 10

Submit a script or a program named code.x and store log files in the output_dir folder

qexec -o output_dir -c code.x --verbose -param 1.0

qbarrier: halt execution until some jobs are over

It provides a simple means to create command-line-level dependencies between jobs. For finer control on job dependencies, see 'job arrays' in the gridengine documentation.

qjobs: jobs' reporting tool

It provides colorized summaries and status listings of running jobs. It assumes that jobs were submitted using qexec, but it will mostly work even if this is not the case. You can tag a list of jobs, to keep track of changes between successive calls.

qjobs -t my_jobs runs/*.job

DONE        44.26%  355 /  802  [+15.21%]
FAIL         5.11%   41 /  802  [+0.00%]
KILL         0.00%    0 /  802  [+0.00%]
OVER        49.38%  396 /  802  [+15.21%]
RUN          1.50%   12 /  802  [+0.62%]
WAIT        49.13%  394 /  802  [-15.84%]

List error messages of jobs that failed, sorting by modification date

qjobs --sort --status fail $(find . -name "*.job")

Installation

The above goodies are plain python and bash scripts. You can just copy them and put them somewhere in your path. However, to keep up with updates it is better to clone the code repository and install the scripts accordingly

git clone https://gitlab.info-ufr.univ-montp2.fr/daniele.coslovich/gridengine-goodies.git
cd gridengine-goodies
make user

The above procedure will put the scripts in ~/.local/bin so make sure it is in your PATH

export PATH=$HOME/.local/bin:$PATH

Add the above line to your .bashrc to make it persistent across sessions.

To update to the latest version

cd gridengine-goodies
git pull
make user

Authors

Daniele Coslovich: http://www.coulomb.univ-montp2.fr/perso/daniele.coslovich/