oar-docker 0.3.0

Manage a small OAR developpement cluster with docker.

Build your own OAR cluster with docker
--------------------------------------

oar-docker is a set of docker images especially configured for deploying
your own OAR cluster. The main idea is to have a mini development cluster with
a frontend, a server and some nodes that launch in just a few seconds on a
simple laptop.


Why use oar-docker ?
--------------------

Various case scenarios may affect you:
- Quickly test OAR on a cluster
- Gain time: a ten-node cluster (or more) is launched in just a
few seconds and is cleaned in less than a second.
- Save resources: docker allows the user to pool the node between
various systems, resource utilization is thus considerably reduced.
- Synced volume : allowing you to continue working on your host machine, but
use the resources in the guest machine to compile or run your project.


Installation
------------

Requirements:
- python 2.7
- docker >= 1.3

You can install, upgrade, uninstall oar-docker with these commands::

$ pip install oar-docker
$ pip install --upgrade oar-docker
$ pip uninstall oar-docker

Or from git (last development version)::

$ pip install git+https://github.com/oar-team/oar-docker.git

Or if you already pulled the sources::

$ pip install path/to/sources

Or if you don't have pip::

$ easy_install oar-docker

Usage
-----

::

Usage: oardocker [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

Manage a small OAR developpement cluster with docker.

Options:
--workdir DIRECTORY Changes the folder to operate on.
--docker-host TEXT The docker socket [default:unix://var/run/docker.sock]
--cgroup-path TEXT The cgroup file system path [default: /sys/fs/cgroup]
--docker-binary TEXT The docker client binary [default: docker]
--version Show the version and exit.
-h, --help Show this message and exit.

Commands:
build Build base images
clean Remove all stopped containers and untagged...
connect Connect to a node.
destroy Stop containers and remove all images
init Initialize a new environment.
install Install and configure OAR from src
logs Fetch the logs of all containers.
ssh Connect to machine via SSH [deprecated]
ssh-config Output OpenSSH valid ssh config [deprecated]
start Create and start the containers
status Output status of the cluster
stop Stop and remove all containers


Getting started
---------------

To get started with oar-docker, the first thing to do is to initialize a
project::

$ oardocker init

If you already have OAR sources, the best is to initialize directly the
oardocker project in the OAR sources directory::

$ cd path/to/oar/src
$ oardocker init

You have to do this only once. It allows you to import the Dockerfiles
and other configuration files.

We then launch the base image build::

$ oardocker build

Now, we have to install OAR. To do this, several options are available.

If you already have the OAR sources::

$ oardocker install . ## ou . est le chemin vers les sources de OAR

Or if you want to install from tarball::

$ oardocker install http://oar-ftp.imag.fr/oar/2.5/sources/testing/oar-2.5.4+rc4.tar.gz

You can also launch the installation from a git repository::

$ oardocker install git+https://github.com/oar-team/oar.git


We start a OAR cluster with 5 nodes::

$ oardocker start -n 5

It is possible to share directories between host machines and
all containers with the ``-v`` option::

$ oardocker start -v $PWD:/oar_src -v /my/custom/lib:/usr/local/ma_lib

To manage the cluster::

$ oardocker connect frontend|server|nodeXX
$ oardocker logs [frontend|server|nodeXX]


To clean::

$ oardocker stop ## stops and removes all containers
$ oardocker clean ## removes all stopped containers (failed) and the untagged images <none:none>
$ oardocker destroy ## removes all images and containers


With oar-docker, it is possible to chain all commands to go faster::

$ oardocker init -f build install oar-2.5.4+rc4.tar.gz start -n 4 connect -l root frontend

For instance, to develop on OAR, we often need to install OAR,
start the cluster and connect to it::


$ oardocker install $PWD start -n 10 -v $PWD:/home/docker/oar_src connect frontend


One last thing to know. The ``stop`` command is automatically launched before
every ``start``, ``install`` and ``build`` ... If we launch multiple times the
last command, we will always obtain the same result. It can be useful to
experiment and develop (even) faster.


Security
--------

oar-docker is a development project and a testing one. It is in no way secure.
Besides, the private ssh key used is also insecured since it is public (you can
find it in the sources).


Related resources
-----------------

- `A minimal Ubuntu base image modified for Docker-friendliness`_
- `Got a Minute? Spin up a Spark cluster on your laptop with Docker`_


oar-docker CHANGELOG
====================

version 0.3.0
=============

Released on Nov 27th 2014

**Features**:

- Added ``oardocker exec`` command
- Manage multiple environment variants with ``oardocker init``: added wheezy|jessie|python bases images

**Bug fixes**:
- Revert default environment to Debian Wheezy due to breaking OAR API in Jessie
- Fixed locales issue

**Improvements**:
- better synchronisation between oar-server and postgresql services


version 0.2.0
=============

Released on Nov 5th 2014

**Features**:

- Updated base images to debian jessie
- Added ``oardocker connect`` to connect to the nodes without ssh
- The commands ``oardocker ssh`` and ``oardocker ssh-config`` are deprecated from now

**Improvements**:

- Removed supervisor and make init process less complex by only using my_init.d statup scripts
- Customized help parameter to accept ``-h`` and ``--help``
- Used docker client binary for some task instead of the API

**Bug fixes**:

- Make sure that /etc/hosts file contain the localhost entry

version 0.1.4
=============

Released on Oct 28th 2014

- Ignored my-init scripts if filename ends by "~"
- Added wait_pgsql script to wait postgresql to be available
- Fixed monika config (db server hostname is server)
- Removed old code
- Adapt cgroup mount script to job_resource_manager_cgroup.pl and remove old cpuset workaround
- Fixed cpu/core/thread affinity


version 0.1.3
=============

Released on Sep 10th 2014

- Added `oar reset` cmd to restart containers
- Added a better comments about oardocker images with git information
- Used default job_resource_manager script (from oar sources)
- Mount the host cgroup path in the containers (default path is /sys/fs/cgroup)
- Removed stopped containers from ssh_config
- Remove dnsmasq and mount a custom /etc/hosts for the nodes (need docker >= 1.2.0)


version 0.1.2
=============

Released on Sep 16th 2014

- Keep compatible with older versions of git
- Don't name the containers
- Mounting OAR src as Copy-on-Write directory with unionfs-fuse
- Stopped installation when container failed during ``oardocker install``
- Added option to print version
- Allow ssh connection with different user

version 0.1.1
=============

Released on Sep 11th 2014

- Minor bug fixes

version 0.1
===========

Released on Sep 11th 2014

Initial public release of oar-docker