clusterdock 2.3.0

clusterdock is a framework for creating Docker-based container clusters

=========== clusterdock

.. image:: https://img.shields.io/pypi/l/clusterdock.svg :target: https://pypi.python.org/pypi/clusterdock

.. image:: https://img.shields.io/pypi/v/clusterdock.svg :target: https://pypi.python.org/pypi/clusterdock

.. image:: https://readthedocs.org/projects/clusterdock/badge/?version=latest :target: https://clusterdock.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status

.. image:: https://img.shields.io/pypi/pyversions/clusterdock.svg :target: https://pypi.python.org/pypi/clusterdock

---

clusterdock is a Python 3 project that enables users to build, start, and manage Docker container-based clusters. It uses a pluggable system for defining new types of clusters using folders called topologies and is a swell project, if I may say so myself.

---

"I hate reading, make this quick."

Before doing anything, install a recent version of Docker_ to your machine and install clusterdock:

.. code-block:: console

$ pip3 install clusterdock

Next, clone a clusterdock topology to your machine. For this example, we'll use the nodebase topology_. You could start a 2-node cluster:

.. _Docker: https://www.docker.com/ .. _nodebase topology: https://github.com/clusterdock/topology_nodebase

.. code-block:: console

$ git clone https://github.com/clusterdock/topology_nodebase.git
$ clusterdock start topology_nodebase
2017-08-03 10:04:18 PM clusterdock.models   INFO     Starting cluster on network (cluster) ...
2017-08-03 10:04:18 PM clusterdock.models   INFO     Starting node node-1.cluster ...
2017-08-03 10:04:19 PM clusterdock.models   INFO     Starting node node-2.cluster ...
2017-08-03 10:04:20 PM clusterdock.models   INFO     Cluster started successfully (total time: 00:00:01.621).

To list cluster nodes:

.. code-block:: console

$ clusterdock ps

For cluster `famous_hyades` on network cluster the node(s) are:
CONTAINER ID     HOST NAME            PORTS              STATUS        CONTAINER NAME          VERSION    IMAGE
a205d88beb       node-2.cluster                          running       nervous_sinoussi        1.3.3      clusterdock/topology_nodebase:centos6.6
6f2825c596       node-1.cluster       8080->80/tcp       running       priceless_franklin      1.3.3      clusterdock/topology_nodebase:centos6.6

To SSH into a node and look around:

.. code-block:: console

$ clusterdock ssh node-1.cluster
[root@node-1 ~]# ls -l / | head
total 64
dr-xr-xr-x   1 root root 4096 May 19 20:48 bin
drwxr-xr-x   5 root root  360 Aug  4 05:04 dev
drwxr-xr-x   1 root root 4096 Aug  4 05:04 etc
drwxr-xr-x   2 root root 4096 Sep 23  2011 home
dr-xr-xr-x   7 root root 4096 Mar  4  2015 lib
dr-xr-xr-x   1 root root 4096 May 19 20:48 lib64
drwx------   2 root root 4096 Mar  4  2015 lost+found
drwxr-xr-x   2 root root 4096 Sep 23  2011 media
drwxr-xr-x   2 root root 4096 Sep 23  2011 mnt
[root@node-1 ~]# exit

To see full usage instructions for the start action, use -h/--help:

.. code-block:: console

$ clusterdock start topology_nodebase -h
usage: clusterdock start [-h] [--node-disks map] [--always-pull]
                         [--namespace ns] [--network nw] [-o sys] [-r url]
                         [--nodes node [node ...]]
                         topology

Start a nodebase cluster

positional arguments:
  topology              A clusterdock topology directory

optional arguments:
  -h, --help            show this help message and exit
  --always-pull         Pull latest images, even if they're available locally
                        (default: False)
  --namespace ns        Namespace to use when looking for images (default:
                        clusterdock)
  --network nw          Docker network to use (default: cluster)
  -o sys, --operating-system sys
                        Operating system to use for cluster nodes (default:
                        centos6.6)
  -r url, --registry url
                        Docker Registry from which to pull images (default:
                        None)

nodebase arguments:
  --node-disks map      Map of node names to block devices (default: None)

Node groups:
  --nodes node [node ...]
                        Nodes of the nodes group (default: ['node-1',
                        'node-2'])

When you're done and want to clean up:

.. code-block:: console

$ clusterdock manage nuke
2017-08-03 10:06:28 PM clusterdock.actions.manage INFO     Stopping and removing clusterdock containers ...
2017-08-03 10:06:30 PM clusterdock.actions.manage INFO     Removed user-defined networks ...

To see full usage instructions for the build action, use -h/--help:

.. code-block:: console

$ clusterdock build topology_nodebase -h
usage: clusterdock build [--network nw] [-o sys] [--repository repo] [-h]
                         topology

Build images for the nodebase topology

positional arguments:
  topology              A clusterdock topology directory

optional arguments:
  --network nw          Docker network to use (default: cluster)
  -o sys, --operating-system sys
                        Operating system to use for cluster nodes (default:
                        None)
  --repository repo     Docker repository to use for committing images
                        (default: docker.io/clusterdock)
  -h, --help            show this help message and exit

======= History

2.3.0 (2020.08.10)

• Ability to upload character or byte content file to Nodes
• Pin docker-py version to 4.2.2

2.2.0 (2019.09.16)

• Clean up console output.
• Add support for specifying env vars in :py:class:clusterdock.models.Node.
• Improved support for CentOS 7.

2.1.0 (2018.06.18)

• Add argument for clusterdock-config-directory.

2.0.4 (2018.06.07)

• Increase Docker client timeout.

2.0.3 (2018.05.29)

• Add SSH daemon check to :py:meth:clusterdock.models.Node.start.

2.0.2 (2018.05.22)

• Add clusterdock labels to volumes_from containers.

2.0.1 (2018.05.18)

• Workaround for /etc/localtime mount failing on Mac.

2.0.0 (2018.04.02)

• Update to work against docker-py > 3.0.0.

1.6.0 (2018.03.19)

• Add --port argument functionality to clusterdock start.

1.5.0 (2018.03.09)

• Add support for build action.
• Use Docker labels for clusterdock nodes and clusters.
• Enhance clusterdock manage action.
• Add clusterdock ps action.
• Add clusterdock cp action.

1.4.0 (2018.02.21)

• Add nodes to /etc/hosts during start.

1.3.3 (2018.02.08)

• Fix docker-py dependency to 2.7.0.

1.3.2 (2017.11.13)

• Added support for executing commands in detached mode.

1.3.1 (2017.11.07)

• Fixed broken fix of volume handling from previous release.

1.3.0 (2017.11.01)

• Fixed handling of duplicate networks.
• Made :py:meth:clusterdock.models.Node.execute run commands in a shell (using /bin/sh by default).
• Fixed handling of volumes passed to :py:class:clusterdock.models.Node.

1.2.0 (2017.10.23)

• Changed return type of :py:meth:clusterdock.models.Cluster.execute and :py:meth:clusterdock.models.NodeGroup.execute.
• Added support for node devices.

1.1.0 (2017.09.21)

• Updated :py:meth:clusterdock.models.Node.execute to return a namedtuple with the command's exit code and output.
• Fixed bug around quiet argument to :py:meth:clusterdock.models.Node.execute.
• Added support for specifying host:container port mappings when creating a node.
• Added ip_address attribute to :py:class:clusterdock.models.Node.

1.0.7 (2017.09.18)

• Removed :py:const:DEFAULT_NAMESPACE to let topologies define their own.

1.0.6 (2017.09.04)

• Added :py:meth:clusterdock.models.Node.put_file and :py:meth:clusterdock.models.Node.get_file.
• Made network an instance attribute of :py:class:clusterdock.models.Cluster.

1.0.5 (2017.09.02)

• Added logic to pull missing images to :py:mod:clusterdock.models.

1.0.4 (2017.09.02)

• Fixed missing install requirement.

1.0.3 (2017.09.02)

• Cleaned up :py:class:clusterdock.models.Node API.
• Added wait_for_permission and join_url_parts utility functions.

1.0.2 (2017.08.04)

• Updated how Cluster and Node objects are initialized.
• Added project logo.
• Doc improvements.

1.0.1 (2017.08.03)

• First release on PyPI.