Skip to main content

Autosubmit: a versatile tool to manage Weather and Climate Experiments in diverse Supercomputing Environments

Project description


Autosubmit is a tool to create, manage and monitor experiments using
Computing Clusters, HPC's and Supercomputers remotely via ssh.


HOW TO DEPLOY/SETUP AUTOSUBMIT FRAMEWORK
========================================

- Autosubmit has been tested:

with the following Operating Systems:
* Linux Debian

on the following HPC's/Clusters:
* Ithaca (IC3 machine)
* Stargate (IC3 machine)
* MareNostrum (BSC machine)
* MareNostrum3 (BSC machine)
* HECToR (EPCC machine)
* Lindgren (PDC machine)
* C2A (ECMWF machine)
* CCA (ECMWF machine)
* ARCHER (EPCC machine)

- Pre-requisties: These packages (bash, python2, sqlite3, git-scm > 1.8.2, subversion) must be available at local
machine. These packages (argparse, dateutil, pyparsing, numpy, pydotplus, matplotlib, paramiko) must be available for
python runtime. And the machine is also able to access HPC's/Clusters via password-less ssh.

- Install Autosubmit
> pip install autosubmit
or download, unpack and "python setup.py install"

- Create a repository for experiments: Say for example "/cfu/autosubmit" then "autosubmit configure" and follow
instructions

- Create a blank database: "autosubmit install"

HOW TO USE AUTOSUBMIT
=====================
> autosubmit -h

> autosubmit expid --HPC ithaca --description "experiment is about..."

Say for example, "cxxx" is 4 character based expid generated by system automatically.
First character is a letter, the other three alfa-numeric characters allow to identify uniquely the experiment.

> vi /cfu/autosubmit/cxxx/conf/expdef_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/autosubmit_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/platforms_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/jobs_cxxx.conf

> autosubmit create cxxx

> nohup autosubmit run cxxx

Cautions:
- Before launching autosubmit check the following stuff:
> ssh ithaca # (for example) check other HPC's where password-less ssh is feasible.
- After launching autosubmit, one must be aware of login expiry limit and policy (if applicable for any HPC)
and renew the login access accordingly (by using token/key etc) before expiry.

HOW TO MONITOR EXPERIMENT
=========================

> autosubmit monitor -h

> autosubmit monitor cxxx
or
> autosubmit monitor cxxx -o png

Above generated plot with date & time stamp can be found at:

/cfu/autosubmit/cxxx/plot/cxxx_date_time.pdf
or
/cfu/autosubmit/cxxx/plot/cxxx_date_time.png


HOW TO RESTART EXPERIMENT
=========================

> autosubmit recovery -h

> autosubmit recovery cxxx -s # saving the pickle file

> nohup autosubmit run cxxx


FULL DOCUMENTATION AND HOW TOs
==============================

Check the Autosubmit documentation provided in the docs/ folder of the package, in PDF format.
Check the online documentation in the following web page: http://www.bsc.es/projects/earthscience/autosubmit/

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

autosubmit-3.2.0b4.tar.gz (1.3 MB view details)

Uploaded Source

File details

Details for the file autosubmit-3.2.0b4.tar.gz.

File metadata

File hashes

Hashes for autosubmit-3.2.0b4.tar.gz
Algorithm Hash digest
SHA256 d735518b16471b1bd62cf65b9423035293f871ac92930e3d48a15b271b69b129
MD5 640074b5af708fb811db856b3809b620
BLAKE2b-256 dea655e477e92c9f8c507317d3f3f67e802f69e061b0ecfdb9a7a507324a4b84

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page