This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Tools for large scale data clustering.

Project Description

Coordinate is a standalone server process and client library for multiple-system coordination and synchronization capabilities. It manages work specifications and work units, weighted scheduling, priority scheduling, and dependency graph.

Configuration and execution

In a global yakonfig config.yaml configuration file, add coordinate at the top level:

coordinate:
  server:
    host: 0.0.0.0
    port: 5932
  namespace: *namespace
  registry_addresses: ['10.0.1.x:5932']

Run coordinated with this configuration, or manually specify --host and --port. The default configuration is to only listen on the localhost interface on port 5932.

The main interface is through the TaskMaster object. This implementation makes RPC calls to the coordinated server.

The server is pluggable and additional functionality can be through other server objects similar to the JobQueue class.

RPC system

Coordinated uses a simple RPC system based on CBOR messages. Client processes initiate a TCP connection as indicated in the configuration and send CBOR messages like (JSON syntax):

{"id": 1,
 "method": "lock",
 "params": [...]}

where id is a sequential number per client object, starting at 1. There is one TCP connection per client object, but this connection is held open until explicitly closed.

The responses look like:

{"id": 1,
 "result": "value"}

Or:

{"id": 2,
 "error": {"message": "No such method"}}

If the server-side call raised an exception, the formatted traceback is included as the error message.

The server supports a CBOR list of request messages, and will return a CBOR list of responses.

Client classes subclass coordinated._cbor_rpc_client.CborRpcClient and call _rpc() to perform a call. This method blocks on the connection and response. If the server returned the error form, this is raised as an exception.

The server is run with a proxy object, presently coordinated.lockd.MultiBackendProxyObject, and calls the requested methods on that object with the requested parameters.

Release History

Release History

History Node

0.8.9.dev2

This version
History Node

0.1.5

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
coordinate-0.1.5.tar.gz (95.8 kB) Copy SHA256 Checksum SHA256 Source Aug 17, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting