pyjobs

A Python-based and jupyter notebook-friendly job manegement system for remote job management systems.

pyjobs allows to submit/list/detele/track jobs on Univa Grid Engine (UGE) or Sun Grid Engine (SGE). It enables to remotely execute qsub, qstat, qdel (via ssh) and show the log/error files on local jupyter notebook (or jupyter lab). It can also make it possible to execute jobs on other machines via ssh and nohup (under developments).

Requirements

ssh environment to remote machine. pyjobs is designed to be run on local machine. ssh connection to remote machines (such as ABCI) must be prerequired.

Usage

pyjobs is designed to be run on local Jupyter notebook environment.

On jupyter notebooks, run

from pyjobs import *
import subprocess,json
jobs=pyjobs()
jobs,type(jobs),json.dumps(jobs)

to initialize jobs instance of pyjobs.

jobs instance is used to manage your jobs on a specific remote (or local) machine. jobs.qsub() jobs.show() jobs.kill() jobs.rm() jobs.bye()

Run job

jobs.qsub(s2o,"s2o.sh",jc="+gpu,g1,72h",n=0,docker="nvcr-pytorch-1903")

Show job list
```
jobs.show(depth=1,no_update=False)
```

Assign a job to the jobs instance later

jobs[3059261]=pyjob(jobid="3059261",jobname="s2o.sh",jobfile=`cat s2o.sh`)
#jobs.dump()

– Delete a latest job and its log

jobs.kill()+jobs.rm()

jobs.bye(-1)

shell support with ssh

def shellrun(commandline,server,cd,ssh_bash_profile=True,nohup=False)

def run_via_ssh(commandline,server,cd,ssh_bash_profile=True,nohup=False)

when nohup=True, commandline shuould be a shell file.

Metadata

Release files for sshjob 0.0.dev49

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sshjob 0.0.dev49
File	Size	Uploaded
sshjob-0.0.dev49.tar.gz	13.7 kB	May 5, 2020	Details

Release files / sshjob-0.0.dev49.tar.gz

Download URL	sshjob-0.0.dev49.tar.gz
Size	13.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`38bf72bf8e49d5891b55d08e0745700d4dec0a367122e96e1ea906ced80421eb`
BLAKE2b-256 checksum How to use checksums	`90760bbb64395d7c6545b34c32b3831fb6214853d19e2615ad56e3851e602c48`
Upload date	May 5, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.0.0 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.7`