Skip to main content

Port of DrQueue to Python. IPython is used for network communication and task management.

Project description

This is a port of DrQueue to Python. IPython is used for network communication and task management. See more information at https://ssl.drqueue.org/redmine/projects/drqueueipython .

DrQueue has been created by Jorge Daza Garcia-Blanes. See http://www.drqueue.org for more information.

Design changes

  • Master runs IPController

  • Slave runs IPEngine

  • Clients can use the IPython.parallel.Client class to talk to IPController

  • No compiling needed anymore. Just Python code.

  • ZMQ, PyZMQ, MongoDB, PyMongo and IPython become dependencies.

  • MongoDB is be used for information storage of tasks, jobs, pools, … .

  • There is no direct access to frame information anymore. Jobs can be devided into tasks. Depending on the blocksize, one task can consist of one or more frames.

  • High water mark (HWM) can be set for IPEngines in order to control how many tasks are queued to each engine and to always keep some tasks for late joining engines.

  • DrQueueIPython provides a Python module for easy accces to the underlying technology. This makes integration into other software which uses Python possible.

Requirements

Installation

Support development

[![Flattr Button](http://api.flattr.com/button/button-static-50x60.png “Flattr This!”)](http://flattr.com/thing/181901/DrQueueIPython-project “DrQueueIPython project”)

License

Copyright (C) 2011 Andreas Schroeder

Licensed under GNU General Public License version 3. See LICENSE for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

DrQueueIPython-0.0.1.tar.gz (18.3 kB view details)

Uploaded Source

File details

Details for the file DrQueueIPython-0.0.1.tar.gz.

File metadata

File hashes

Hashes for DrQueueIPython-0.0.1.tar.gz
Algorithm Hash digest
SHA256 09bb4bbd8600f26ab4abcf82e8e00b5a8a5f21c4ff7d98b1c8183c6378adbb95
MD5 0caeff0e8a9c2da06e53a85560cc977b
BLAKE2b-256 3668435a436c2adf3ae547177c9ccad370e244426bf6bc200e72804f59d708f7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page