Autosubmit: a versatile tool to manage Weather and Climate Experiments in diverse Supercomputing Environments
Autosubmit is a tool to create, manage and monitor experiments using
Computing Clusters, HPC's and Supercomputers remotely via ssh.
HOW TO DEPLOY/SETUP AUTOSUBMIT FRAMEWORK
- Autosubmit has been tested:
with the following Operating Systems:
* Linux Debian
on the following HPC's/Clusters:
* Ithaca (IC3 machine)
* Stargate (IC3 machine)
* MareNostrum (BSC machine)
* MareNostrum3 (BSC machine)
* HECToR (EPCC machine)
* Lindgren (PDC machine)
* C2A (ECMWF machine)
* CCA (ECMWF machine)
* ARCHER (EPCC machine)
- Pre-requisties: These packages (bash, python2, sqlite3, git-scm > 1.8.2, subversion) must be available at local
machine. These packages (argparse, dateutil, pyparsing, numpy, pydotplus, matplotlib, paramiko) must be available for
python runtime. And the machine is also able to access HPC's/Clusters via password-less ssh.
- Install Autosubmit
> pip install autosubmit
or download, unpack and "python setup.py install"
- Create a repository for experiments: Say for example "/cfu/autosubmit" then "autosubmit configure" and follow
- Create a blank database: "autosubmit install"
HOW TO USE AUTOSUBMIT
> autosubmit -h
> autosubmit expid --HPC ithaca --description "experiment is about..."
Say for example, "cxxx" is 4 character based expid generated by system automatically.
First character is a letter, the other three alfa-numeric characters allow to identify uniquely the experiment.
> vi /cfu/autosubmit/cxxx/conf/expdef_cxxx.conf
> vi /cfu/autosubmit/cxxx/conf/autosubmit_cxxx.conf
> vi /cfu/autosubmit/cxxx/conf/platforms_cxxx.conf
> vi /cfu/autosubmit/cxxx/conf/jobs_cxxx.conf
> autosubmit create cxxx
> nohup autosubmit run cxxx
- Before launching autosubmit check the following stuff:
> ssh ithaca # (for example) check other HPC's where password-less ssh is feasible.
- After launching autosubmit, one must be aware of login expiry limit and policy (if applicable for any HPC)
and renew the login access accordingly (by using token/key etc) before expiry.
HOW TO MONITOR EXPERIMENT
> autosubmit monitor -h
> autosubmit monitor cxxx
> autosubmit monitor cxxx -o png
Above generated plot with date & time stamp can be found at:
HOW TO RESTART EXPERIMENT
> autosubmit recovery -h
> autosubmit recovery cxxx -s # saving the pickle file
> nohup autosubmit run cxxx
FULL DOCUMENTATION AND HOW TOs
Check the Autosubmit documentation provided in the docs/ folder of the package, in PDF format.
Check the online documentation in the following web page: http://www.bsc.es/projects/earthscience/autosubmit/
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.