execo 2.7

Python API for asynchronous control of local or remote, standalone or parallel, unix processes.

Execo offers a Python API for asynchronous control of local or remote, standalone or parallel, unix processes. It is especially well suited for quickly and easily scripting workflows of parallel/distributed operations on local or remote hosts: automate a scientific workflow, conduct computer science experiments, perform automated tests, etc. The core python package is execo. The execo_g5k package provides a set of tools and extensions for the Grid5000 testbed. The execo_engine package provides tools to ease the development of computer sciences experiments.

License

Execo is copyright INRIA Rhone-Alpes, Service Experimentation et Developpement.

Execo is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Execo is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with Execo. If not, see <http://www.gnu.org/licenses/>

Versions

latest stable version: v2.7 (2023-01-25)

Installation instructions

useful links

• home page: https://gitlab.inria.fr/mimbert/execo

• documentation pages: https://mimbert.gitlabpages.inria.fr/execo/

• package download: https://gitlab.inria.fr/mimbert/execo/-/packages

• mailing lists: https://sympa.inria.fr/sympa/info/execo-users

• code repository: [e-mail]:mimbert/execo.git / https://gitlab.inria.fr/mimbert/execo.git

platforms

• works on linux (primary development platform), darwin (macosx)

• should work on bsd (not tested)

• don’t know on windows or cygwin (not tested)

prerequisites

execo installation absolutely requires python 2.6 / 2.7 or >=3.2. execo_g5k needs python-requests. Optionnal packages are (debian package names, in order of decreasing importance):

• under python 2.6, python-argparse is needed for execo_engine.engine.Engine

• python-keyring for allowing storage of execo_g5k.api_utils api password in the desktop environment keyring (allowing asking it only once).

• python-psycopg2 for optimized interface to Grid5000 OAR planning.

• sphinx-doc, graphviz for building the documentation (usually not needed for regular users).

• python-matplotlib (>= 1.2.0) for some graphical representations.

• python-networkx, python-pygraphviz, graphviz for module execo_g5k.topology. Later versions of networkx (at least v1.11) also require python-pydotplus, but this dependency is not explicitely listed in the debian package (as of July 2017).

At runtime, connecting to remote hosts requires ssh, scp or similar connection tools, and optionnaly taktuk (probably >= 3.6. http://taktuk.gforge.inria.fr/). execo.action.ChainPut requires bourne shell and netcat on remote hosts.

installation

To install execo from a source package (.tar.gz) or from the git tree:

$ python setup.py install [--user]

to build documentation (if sphinx is available):

$ python setup.py build_doc

to install documentation (if sphinx is available):

$ python setup.py install_doc

It is possible to install execo automatically with pip or easy_install:

$ pip install --user execo

or:

$ easy_install --user execo

It is possible to generate a debian package. For example, in the execo package toplevel dir, run (the .deb will be generated in ../):

$ dpkg-buildpackage -us -uc

Usage

See html documentation for module execo and execo_g5k at https://mimbert.gitlabpages.inria.fr/execo/

Bugs

Execo is regularly used to perform advanced experiments and administration / monitoring tasks, in and outside Grid5000. We actively fix bugs. Bugs should be reported to http://bugzilla.inria.fr (product: execo)

Publications

Matthieu Imbert, Laurent Pouilloux, Jonathan Rouzaud-Cornabas, Adrien Lèbre, Takahiro Hirofuchi “Using the EXECO toolbox to perform automatic and reproducible cloud experiments” 1st International Workshop on UsiNg and building ClOud Testbeds UNICO, collocated with IEEE CloudCom 2013 2013

How to contribute

• start contributing by sending clean patches or report bugs.

• stay consistent with the coding and naming style

• code must be compatible with python 2.6, 2.7 and 3.2+ (see https://mimbert.gitlabpages.inria.fr/execo/python2-python3.html)

• use the core systems provided (eg. the configuration system, the tty coloring system, the logger, etc.) instead of developing your own.

• provide documented code (internal documentation as well as user documentation where needed)

• ask a core developer before adding a dependency or dealing with threads, signals (particularly: creating threads)

• indent with spaces, not tabs. one level of indentation is four spaces (if needed: use reindent.py -rnv . in execo top directory)

• recommended commit messages format:

  [<module_name>] <category>: commit message.

  • <module_name> can be execo, execo_g5k, execo_engine. Omit if commit is not specific to a module.

  • <category> is free form but should indicate which part(s) of the module the commit affects. Omit if a commit is not specific to a module part.