Uploads SLURM job information to WandB for better monitoring.
Project description
slurm-to-wandb
Monitor SLURM jobs using Weights and Biases (wandb) 📊
Usage
On your cluster, login to wandb using the cli, then run:
pip install slurm-to-wandb
slurm_to_wandb <job_ids>
... to upload all sacct
information to wandb. Currently, uploads to a project called "peregrine" - support for configuring this is planned #4. <job_ids>
can be multiple space-separated job ids. In the case multiple job id's match, all matched job ids are uploaded (useful, for example, when using job arrays).
To construct a DataFrame with the information yourself, use the function slurm_to_wandb.sacct_as_df(*job_ids, **sacct_args)
. sacct_args
can be any additional args to pass to sacct
. Run it on the cluster:
from slurm_to_wandb import sacct_as_df
df = sacct_as_df("job_id_123", "another_job_id", starttime="2021-05-20")
df
You can now upload the information to wandb however you like 🙌🏻
About
By Jeroen Overschie.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for slurm_to_wandb-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51acd4cf88f9e99b7b6f46ed0f0f8102707e9b7f8cc88f8771713c0ab3928b19 |
|
MD5 | 79d7d5deb98dee6ad0f32c27b47100a9 |
|
BLAKE2b-256 | e4f724486c8171a4200e739f7d1222db3bec6fbc99535022d47da437a5d0e154 |