easybuild 1.8.1

EasyBuild is a software installation framework in Python that allows you to install software in a structured and robust way.

EasyBuild is a software build and installation framework written in Python that allows you to install software in a structured, repeatable and robust way.

It is motivated by the need for a tool that allows to:

• independently install multiple versions of software side-by-side

• support multiple compilers and libraries for building software and its dependencies

• keep the software build configuration simple

• divert from the standard configure / make / make install with custom procedures (which is often necessary for scientific software)

• use environment modules for dependency resolution and making the software available to users in a transparent way

• keep record of the installation logs

• keep track of installation configuration in version control

Some key properties of EasyBuild:

• installation configuration is done using a (very concise) .eb specification file

• custom behaviour is described in easyblocks; these are Python classes that can be plugged into the EasyBuild framework

• the generation of the module files to easily make the software available to users

• the dependencies for installation are resolved using environment modules and can be automatically installed using the robot feature

• after the installation, the specification files can be sent to a repository for archiving

For more information on EasyBuild, see the EasyBuild wiki on GitHub.

Requirements

EasyBuild requires Python 2.4 (or a more recent 2.x version) to be available, as well as the environment modules tool.

The GitPython Python module is recommended, especially when EasyBuild is being used from a git repository.

See the EasyBuild wiki for more information on EasyBuild dependencies.

Availability

EasyBuild is split up into three different packages, which are available from the Python Package Index (PyPi):

• easybuild-framework - the EasyBuild framework, which includes the easybuild.framework and easybuild.tools Python packages that provide general support for building and installing software

• easybuild-easyblocks - a collection of easyblocks that implement support for building and installing (collections of) software packages

• easybuild-easyconfigs - a collection of example easyconfig files that specify which software to build, and using which build options; these easyconfigs will be well tested with the latest compatible versions of the easybuild-framework and easybuild-easyblocks packages

Next to these packages, a meta-package named easybuild to easily install the full EasyBuild distribution is also available, see PyPi.

The source code for these packages is available on GitHub, see easybuild-framework sources, easybuild-easyblocks sources, easybuild-easyconfigs sources.

Installation

Installing EasyBuild is really easy: just run

easy_install --user easybuild

or

easy_install --prefix=/some/dir easybuild

to install the latest available version of all three EasyBuild packages.

You can also download the tarballs for the EasyBuild packages yourself, unpack them and run

python setup.py install --prefix=/some/dir

for each of the packages.

Do make sure you set your PYTHONPATH before performing the installation procedure (if not, easy_install or setup.py will complain).

Do not forget to also update your PATH environment variable, to include “/some/dir/bin” such that the main ‘eb’ program will be available.

Quick demo for the impatient

To see EasyBuild in action, install it and build HPL with the robot feature of EasyBuild, by running the following commands:

easy_install --user easybuild
export PATH=$PATH:$HOME/.local/bin
eb HPL-2.0-goalf-1.1.0-no-OFED.eb --robot

This will install EasyBuild to your home directory ($HOME/.local), update the PATH environment variable such that the ‘eb’ command is available and subsequently build and install HPL, after building and installing a GCC-based compiler toolchain and all of its dependencies using the default EasyBuild configuration, which will install to $HOME/.local/easybuild/software.

The entire process should take about an hour on a recent system.

Module files will be provided in $HOME/.local/easybuild/modules/all, so to load the provided modules, update your MODULEPATH environment variable.

Notes:

• next to Python and the environment-modules package, a C and C++ compiler are required on your system, e.g., gcc and g++

• you will needs a coupe of GBs of free space in your home directory (mainly due to the GCC build)

• the installation ATLAS library, which is part of this demo, may fail on a virtual machine due to the self-tuning install mechanism

• if the –user option for easy_install is not available on your system, you can use the following commands instead:

PYLIB=`python -c "import distutils.sysconfig; print distutils.sysconfig.get_python_lib(prefix='$HOME/.local'); "`
mkdir -p $PYLIB
export PYTHONPATH=$PYLIB:$PYTHONPATH
easy_install --prefix=$HOME/.local easybuild
export PATH=$PATH:$HOME/.local/bin
eb HPL-2.0-goalf-1.1.0-no-OFED.eb --robot

Quick start

To get started, you first need to configure EasyBuild for use.

Once this is done, using EasyBuild is as simple as creating a .eb specification file, and providing it to the framework:

eb example.eb

For command line options, see

eb -h (or --help)

See the EasyBuild wiki for documentation on writing your own easyconfig files (.eb).

To add support for particular software that requires a custom installation procedure, you will need to implement an easyblock that can be plugged into the EasyBuild framework (see Development guide).

On the EasyBuild wiki, a step-by-step guide to getting started with EasyBuild is provided.

Contact info

You can get in contact with the EasyBuild community in different ways:

Mailing list

An EasyBuild mailinglist [e-mail] is available to subscribe to.

This list is used by both users and developers of EasyBuild, so if you have any questions or suggestions, you can post them there.

Only members can post to this mailinglist. To request membership, see https://lists.ugent.be/sympa/info/easybuild.

IRC

An IRC channel #easybuild has been set up on the FreeNode network.

Just connect your IRC client to the irc.freenode.net server, and join the #easybuild channel.

There is an IRC bot present (easybuilder). Just type !help to get pointers to the available commands.

Twitter

The EasyBuild team also has a Twitter feed: @easy_build.

Disclaimer

EasyBuild has mainly been tested on RPM-based 64-bit Linux systems, i.e., Scientific Linux 5.x/6.x. Support for other Linux distributions and operating systems is pending.

License

EasyBuild is developed by the High-Performance Computing team at Ghent University and is made available under the GNU General Public License (GPL) version 2.

Acknowledgements

EasyBuild was created with support of Ghent University , the Flemish Supercomputer Centre (VSC) , the Hercules foundation and the Department of Economy, Science and Innovation (EWI) .