Skip to main content

Read Slurm sacct as pandas dataframes

Project description

Pandas sacct

Provides an Sacct class interface to the Slurm.

In general options to sacct (see man sacct) can passed by keyword to Sacct.

Usage

  • Sacct(flag=True) corresponds to sacct --flag.
  • Sacct(key=value) corresponds to sacct --key=value.
  • starttime and endtime options support both string and types that support .isoformat() such as pd.Timestamp().

examples

import sacct
s = sacct.Sacct(starttime="2020-08-30T00:00:00", endtime="2020-08-31T00:00:00", allocations=True)
df = s.execute()
print(s)
print(df.head())
sacct --format=jobidraw,start,end,nodelist --noheader --parsable2 --truncate --allocations --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00
   jobidraw               start                 end    nodelist
0  33891896 2020-08-30 03:01:58 2020-08-30 03:15:14  [nid01454]
1  33891898 2020-08-30 03:01:58 2020-08-30 03:04:59  [nid01565]
2  33891899 2020-08-30 03:07:13 2020-08-30 03:10:10  [nid01163]
3  33891900 2020-08-30 03:01:58 2020-08-30 03:15:56  [nid01998]
4  33891902 2020-08-30 03:01:58 2020-08-30 03:04:54  [nid01716]
import sacct
import pandas as pd
t0 = pd.Timestamp("2020-08-30T00:00:00")
t1 = t0 + pd.Timedelta(days=1)
s = sacct.Sacct(starttime=t0, endtime=t1, allocations=True, format=['jobidraw', 'jobname'], allusers=True)
df = s.execute()
print(s)
print(df.head())
sacct --format=jobidraw,jobname --noheader --parsable2 --truncate --allocations --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00 --allusers
   jobidraw   jobname
0  27731125  JT_D1_16
1  27731129  JT_D1_16
2  27731131  JT_D1_16
3  27731136  JT_D1_16
4  27731165  JT_D2_16
import sacct
import pandas as pd
t0 = pd.Timestamp("2020-08-30T00:00:00")
t1 = t0 + pd.Timedelta(days=1)
s = sacct.Sacct(starttime=t0, endtime=t1, format=['jobidraw', 'jobname'], allusers=True)
df = s.execute()
print(s)
print(df.head())
sacct --format=jobidraw,jobname --noheader --parsable2 --truncate --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00 --allusers
   jobidraw    step   jobname
0  29056157   batch     batch
1  29056157  extern    extern
2  29056157       0  lmp_cori
3  29056168   batch     batch
4  29056168  extern    extern

Specific sacct command (optional):

s = sacct.Sacct(sacct_cmd="/usr/bin/sacct", starttime="2020-08-30T00:00:00", endtime="2020-08-31T00:00:00", allocations=True)

Copywrite Notice

Pandas-sacct Copyright (c) 2020, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.

If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Intellectual Property Office at IPO@lbl.gov.

NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit others to do so.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pandas-sacct-1.2.0.tar.gz (4.1 kB view hashes)

Uploaded Source

Built Distribution

pandas_sacct-1.2.0-py3-none-any.whl (5.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page