Skip to main content

Autosubmit: a versatile tool to manage Weather and Climate Experiments in diverseSupercomputing Environments

Project description


Autosubmit is a tool to create, manage and monitor experiments using
Computing Clusters, HPC's and Supercomputers remotely via ssh.


HOW TO DEPLOY/SETUP AUTOSUBMIT FRAMEWORK
========================================

- Autosubmit has been tested:

with the following Operating Systems:
* Linux Debian

on the following HPC's/Clusters:
* Ithaca (IC3 machine)
* Stargate (IC3 machine)
* MareNostrum (BSC machine)
* MareNostrum3 (BSC machine)
* HECToR (EPCC machine)
* Lindgren (PDC machine)
* C2A (ECMWF machine)
* CCA (ECMWF machine)
* ARCHER (EPCC machine)

- Pre-requisties: These packages (bash, python2, sqlite3, git-scm > 1.8.2, subversion) must be available at local
machine. These packages (argparse, dateutil, pyparsing, numpy, pydotplus, matplotlib, paramiko) must be available for
python runtime. And the machine is also able to access HPC's/Clusters via password-less ssh.

- Install Autosubmit
> pip install autosubmit
or download, unpack and "python setup.py install"

- Create a repository for experiments: Say for example "/cfu/autosubmit" then "autosubmit configure" and follow
instructions

- Create a blank database: "autosubmit install"

HOW TO USE AUTOSUBMIT
=====================
> autosubmit -h

> autosubmit expid --HPC ithaca --description "experiment is about..."

Say for example, "cxxx" is 4 character based expid generated by system automatically.
First character is a letter, the other three alfa-numeric characters allow to identify uniquely the experiment.

> vi /cfu/autosubmit/cxxx/conf/expdef_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/autosubmit_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/platforms_cxxx.conf

> vi /cfu/autosubmit/cxxx/conf/jobs_cxxx.conf

> autosubmit create cxxx

> nohup autosubmit run cxxx

Cautions:
- Before launching autosubmit check the following stuff:
> ssh ithaca # (for example) check other HPC's where password-less ssh is feasible.
- After launching autosubmit, one must be aware of login expiry limit and policy (if applicable for any HPC)
and renew the login access accordingly (by using token/key etc) before expiry.

HOW TO MONITOR EXPERIMENT
=========================

> autosubmit monitor -h

> autosubmit monitor cxxx
or
> autosubmit monitor cxxx -o png

Above generated plot with date & time stamp can be found at:

/cfu/autosubmit/cxxx/plot/cxxx_date_time.pdf
or
/cfu/autosubmit/cxxx/plot/cxxx_date_time.png


HOW TO RESTART EXPERIMENT
=========================

> autosubmit recovery -h

> autosubmit recovery cxxx -s # saving the pickle file

> nohup autosubmit run cxxx


FULL DOCUMENTATION AND HOW TOs
==============================

Check the Autosubmit documentation provided in the docs/ folder of the package, in PDF format.
Check the online documentation in the following web page: http://autosbumit.ic3.cat

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

autosubmit-3.1.6.tar.gz (1.3 MB view details)

Uploaded Source

File details

Details for the file autosubmit-3.1.6.tar.gz.

File metadata

  • Download URL: autosubmit-3.1.6.tar.gz
  • Upload date:
  • Size: 1.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for autosubmit-3.1.6.tar.gz
Algorithm Hash digest
SHA256 a39eb32b5da157b68d21ac1a4b93e2e0ed6f6a314779f31cae8099fa8558d5d6
MD5 9767aaa03608362d244f6240ee5d5e29
BLAKE2b-256 fc6dccc485f9a7de29c6b049df8fbe9117a9f34187b5e45322efbe0951877ee1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page