zc.zkzeo 1.0.1

ZEO ZooKeeper

Managing addresses, and especially ports is a drag. ZooKeeper can be used as a service registry. Servers can register themselves and clients can find services there. The zc.zkzeo package provides support for registering ZEO servers and a ZEO client storage that gets addresses from ZooKeeper.

Contents

• Running ZEO servers

  • Including a zc.monitor monitoring server

  • Monitoring

• Defining ZEO clients

  • Defining ZEO clients with Python

    • Database and connection convenience functions

  • Defining ZEO clients in configuration files

• Change History

  • 1.0.1 (2015-01-11)

  • 1.0.0 (2015-01-11)

  • 0.3.2 (2012-07-10)

  • 0.3.1 (2012-06-26)

  • 0.3.0 (2012-02-07)

  • 0.2.1 (2011-12-14)

  • 0.2.0 (2011-12-13)

  • 0.1.1 (2011-12-12)

  • 0.1.0 (2011-12-11)

Running ZEO servers

To run a ZEO server, and register it with ZooKeeper, first create a ZEO configuration file:

<zeo>
   address 127.0.0.1
</zeo>

<zookeeper>
   connection zookeeper.example.com:2181
   path /databases/demo
</zookeeper>

<filestorage>
   path demo.fs
</filestorage>

The ZEO configuration file has the same options as usual, plus a zookeeper section with two options:

connection

A ZooKeeper connection string. This is typically a list of HOST:PORT pairs separated by commas.

path

The path at which to register the server. The path must already exist. When the server starts, it will register itself by creating a subnode of the path with a name consisting of it’s address.

(You can also specify a ZooKeeper session timeout, in milliseconds, with a session-timeout option.)

When specifying the ZEO address, you can leave off the port and the operating system will assign it for you.

To start the server, use the zkrunzeo script:

$ bin/zkrunzeo -C FILENAME

where FILENAME is the name of the configuration file you created.

Including a zc.monitor monitoring server

The zc.monitor package provides a simple extensible command server for gathering monitoring data or providing run-time control of servers. If zc.monitor is in the Python path, zc.zkzeo can start a monitor server and make it’s address available as the monitor property of of a server’s ephemeral port. To request this, we use a monitor-server option in the zookeeper section:

<zeo>
   address 127.0.0.1
</zeo>

<zookeeper>
   connection zookeeper.example.com:2181
   path /databases/demo
   monitor-server 127.0.0.1
</zookeeper>

<filestorage>
   path demo.fs
</filestorage>

The value is the address to listen on.

With the configuration above, if we started the server and looked at the ZooKeeper tree for ‘/databases/demo’ using the zc.zk package, we’d see something like the following:

>>> zk.print_tree('/databases/demo')
/demo
  /127.0.0.1:64211
    monitor = u'127.0.0.1:11976'
    pid = 5082

You can also specify a unix-domain socket name:

<zeo>
   address 127.0.0.1
</zeo>

<zookeeper>
   connection zookeeper.example.com:2181
   path /databases/demo
   monitor-server ./monitor.sock
</zookeeper>

<filestorage>
   path demo.fs
</filestorage>

When using a unix-domain socket, the monitor address isn’t included in the tree:

>>> zk.print_tree('/databases/demo')
/demo
  /127.0.0.1:64213
    pid = 5082

Some notes on the monitor server:

• A monitor server won’t be useful unless you’ve registered some command plugins.

• zc.monitor isn’t a dependency of zc.zkzeoc and won’t be in the Python path unless you install it.

Monitoring

The zkzeo package provides a Nagios plugin. The plugin takes a ZooKeeper connection string and path to look up a ZEO server at (using the zc.zk service-registry framework). For example, to monitor the server defined above:

zkzeo-nagios zookeeper.example.com:2181 /databases/demo

The zkzeo nagios monitor supports the same options as the ZEO nagios monitor, so for example to get full metrics:

zkzeo-nagios -m -s statusfile zookeeper.example.com:2181 /databases/demo

Sometimes, there may be multiple servers registered at the same path, for example if servers are replicated. When monitoring a single server, you need to know which one to check. If you’ve a monitor-server for your ZEO process, as we did above, then you can use that to determine which one to use. Just provide the monitor server address:

zkzeo-nagios -m -M ./monitor.sock zookeeper.example.com:2181 /databases/demo

There’s also a helper function useful for other monitors:

>>> import zc.zkzeo.nagios
>>> [zc.zkzeo.nagios.find_server(
...     'zookeeper.example.com:2181',
...     '/databases/demo',
...     None)] == zk.get_children('/databases/demo')
True
>>> [zc.zkzeo.nagios.find_server(
...     'zookeeper.example.com:2181',
...     '/databases/demo',
...     './monitor.sock')] == zk.get_children('/databases/demo')
True

Defining ZEO clients

You can define a client in two ways, from Python and using a configuration file.

Defining ZEO clients with Python

From Python, use zc.zkzeo.client:

>>> import zc.zkzeo
>>> client = zc.zkzeo.client(
...     'zookeeper.example.com:2181', '/databases/demo',
...     max_disconnect_poll=1)

You pass a ZooKeeper connection string and a path. The Client constructor will create a client storage with addresses found as sub-nodes of the given path and it will adjust the client-storage addresses as nodes are added and removed as children of the path.

You can pass all other ZEO.ClientStorage.ClientStorage arguments, except the address, as additional positional and keyword arguments.

Database and connection convenience functions

You’re usually not really interested in getting a storage object. What you really want is a database object:

>>> db = zc.zkzeo.DB(
...     'zookeeper.example.com:2181', '/databases/demo',
...     max_disconnect_poll=1)

or often, just a database connection:

>>> conn = zc.zkzeo.connection(
...     'zookeeper.example.com:2181', '/databases/demo',
...     max_disconnect_poll=1)

Defining ZEO clients in configuration files

In configuration files, use a zkzeoclient storage section:

%import zc.zkzeo

<zodb>
   <zkzeoclient>
      zookeeper zookeeper.example.com:2181
      server /databases/demo
      max-disconnect-poll 1
   </zkzeoclient>
</zodb>

The options for zkzeoclient are the same as for the standard ZODB zeoclient section, except:

• There’s an extra required zookeeper option used to provide a ZooKeeper connection string.

• There can be only one server option and it is used to supply the path in ZooKeeper where addresses may be found.

Change History

1.0.1 (2015-01-11)

Fixed packaging problem (of course).

1.0.0 (2015-01-11)

• Updated to work with ZEO/ZODB rather than ZODB3.

• Added a Nagios monitoring plugin, the script zkzeo-nagios

0.3.2 (2012-07-10)

• Fixed: Didn’t work with explicit configuration of port 0, which is recently supported by ZConfig.

0.3.1 (2012-06-26)

• Fixed: setting a monitor server on a unix-domain socket didn’t work.

0.3.0 (2012-02-07)

• Added a static extra to force a dependency on zc-zookeeper-static.

• In test mode, use a shorter asyncore loop timeout to make the server shut down faster.

• Fixed: zc.zkzeo depended on zc.zk [static], which forced installation of zc-zookeeper-static, which should be optional.

• Fixed: tests didn’t pass with a recent change in handling of registration with empty host names in zc.zk.

• Fixed: Packaging: distribute can’t install distributions with symlinks, so stopped using symlinks in distribution.

0.2.1 (2011-12-14)

• Fixed bug: The path key on the zookeeper server-configuration section was required, and shouldn’t have been.

0.2.0 (2011-12-13)

• Register the host name from the ZEO address setting with ZooKeeper. (This is often an empty string, which zc.zk turns into the fully-quelified domain name.)

• Fixed bug in handling the monitor-server. The actuall address setting was ignored.

0.1.1 (2011-12-12)

• Fixed a packaging bug.

0.1.0 (2011-12-11)

Initial release.