Skip to main content

To collect SGE job information with a damemon

Project description


1 Introduction

To collect SGE job information with a damemon. see

2 Installation

Make sure your pip is from Python3

$ pip install sgejob

There will be a command sgejob_daemon created under the same directory where your pip command located.

If you want to learn more about Python3 and pip, please refer to and

3 Usage

$ sgejob_daemon

    usage: [-h] [-qstat <str>] [-user <str>] [-PIDFILE <file>]
                            [-DAEMON_LOG <file>] [-sge_infor_file <file>]
                            [-sge_usage_file <file>] [-interval INTERVAL]
                            {start,stop} ...

    Start and stop a daemon to collect SGE job information. see

    Copyright (c) 2018 Guanliang Meng (see

    The function `daemonize()` part was copied from
    which is licensed under the Apache License, Version 2.0.

    positional arguments:
        start               start the daemon
        stop                stop the daemon

    optional arguments:
      -h, --help            show this help message and exit
      -qstat <str>          the qstat command to be used [qstatt]
      -user <str>           whose SGE jobs to be collected? [mengguanliang]
      -PIDFILE <file>       set PIDFILE path [/home/mengguanliang/]
      -DAEMON_LOG <file>    set DAEMON_LOG path
      -sge_infor_file <file>
                            set sge infor_file path
      -sge_usage_file <file>
                            set sge usage_file path
      -interval INTERVAL    how often to check the SGE job status? [300]

using as a module

  In [5]: from sgejob import UserJobs, SgeJob, record_jobs

  In [6]: UserJobs?
  Init signature: UserJobs(user=None, qstat='qstat')
  `UserJobs` object has one attribute `jobs`, which is a dictionary, whose keys are the sge job numbers.

  Then `[job_number]` is also a dictionary, whic has following

  ['JB_job_number', 'JAT_prio', 'JB_name', 'JB_owner', 'state',
   'JAT_start_time', 'queue_name', 'slots']
  File:           ~/soft/script/sgejob/source/v0.0.1/sgejob/
  Type:           type

  In [7]: SgeJob?
  Init signature: SgeJob(job_number=None, qstat='qstat')
  `SgeJob` object has two attributes: `infor` and `usage`.

  `SgeJob.infor` is a dictionary, corresponding to the output content of
  `qstat -j job_number`, where the first column is the key, the second column
  is the value.

  `SgeJob.usage` is also a dictionary, whose content is from
  `SgeJob.infor['usage         1']`.

  `SgeJob.infor` has following keys:

  ['current_time', 'job_number', 'exec_file', 'submission_time', 'owner',
   'uid', 'group', 'gid', 'sge_o_home', 'sge_o_log_name', 'sge_o_path',
   'sge_o_shell', 'sge_o_workdir', 'sge_o_host', 'account', 'cwd',
   'hard resource_list', 'mail_list', 'notify', 'job_name', 'jobshare',
   'hard_queue_list', 'env_list', 'script_file', 'project', 'binding',
   'job_type', 'usage         1', 'binding       1', 'scheduling info']

  `SgeJob.usage` has following keys:

  ['current_time', 'job_number', 'cpu', 'mem', 'io', 'vmem', 'maxvmem']
  File:           ~/soft/script/sgejob/source/v0.0.1/sgejob/
  Type:           type

  In [8]: record_jobs?
  Signature: record_jobs(user=None, qstat='qstat', running_only=True, infor_df_csvfile=None, usage_df_csvfile=None)
  Collect job information and append to infor_df_csvfile and

  user=None, collect current user's jobs.

  if running_only=True, it will only collect the running jobs.

  return: None.
  File:      ~/soft/script/sgejob/source/v0.0.1/sgejob/
  Type:      function

5 Citations

Currently, I have no plan to publish sgejob.

6 Author

Guanliang MENG.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sgejob-0.0.8.tar.gz (20.4 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page