Distributed job execution application built for Galaxy (http://galaxyproject.org/).
This project is a Python server application that allows a Galaxy server to run jobs on remote systems (including Windows) without requiring a shared mounted file systems. Unlike traditional Galaxy job runners - input files, scripts, and config files may be transferred to the remote system, the job is executed, and the results are transfered back to the Galaxy server - eliminating the need for a shared file system.
Full documentation for the project can be found on Read The Docs.
Galaxy job runners are configured in Galaxy’s job_conf.xml file. Some small examples of how to configure this can be found here, but be sure to checkout job_conf.xml.sample_advanced in your Galaxy code base or on Github for complete information.
Full details on different ways to install Pulsar can be found in the install section of the documentaiton, but if your machine has the proper Python dependencies available it can be quickly download and a test job run with.
mkdir pulsar cd pulsar virtualenv venv . venv/bin/activate # .venv\Scripts\activate if Windows pip install pulsar-app pulsar-config pulsar --daemon # just pulsar if Windows pulsar-check # runs a test job
The configuration documentation has many details on securing your Pulsar server and enabling advanced features such as cluster integration and message queue communication.
Development and Testing
The recommended approach to setting up a development environment for Pulsar on Linux or Mac OS X is rougly as follows:
git clone https://github.com/galaxyproject/pulsar cd pulsar virtualenv .venv . .venv/bin/activate # .venv\Scripts\activate if Windows pip install -r requirements.txt pip install -r dev-requirements.txt
This project is distributed with unit and integration tests (many of which will not run under Windows), the following command will install the needed python components to run these tests. The following command will then run these tests:
The following command will then produce a coverage report corresponding to this test and place it in the coverage_html_report subdirectory of this project.:
Checkout the Contributing documentation for many more details on developing and contributing to Pulsar.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
- Allow cURL downloader to resume transfers during staging in (thanks to @natefoo). 0c61bd9
- Fix to cURL downloaders status code handling (thanks to @natefoo). 86f95ce
- Fix non-wheel installs from PyPI. Issue 72
- Fix mesos imports for newer versions of mesos (thanks to @kellrott). fe3e919
- More, better logging. 2b3942d, fa2b6dc
- Changed the name of project to Pulsar, moved to Github.
- New RESTful web services interface.
- SCP and Rsync file staging options added by Eric Rasche. Pull Request
- Allow YAML based configuration.
- Support for more traditional pip/setup.py-style installs.
- Dozens of smaller bugfixes and documentation updates.
- Last version named the LWR - found on BitBucket.
- Still supported in Galaxy as of 15.03 the release.
- Introduced support for submitting to various queueing systems, operation as a Mesos framework, Docker support, and various other advanced deployment options.
- Message queue support.
- Framework for configurable file actions introduced.
- Simple support for running jobs managed by the Python LWR web process.
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size & hash SHA256 hash help||File type||Python version||Upload date|
|pulsar_app-0.5.0-py2.7.egg (499.6 kB) Copy SHA256 hash SHA256||Egg||2.7|
|pulsar_app-0.5.0-py2-none-any.whl (213.4 kB) Copy SHA256 hash SHA256||Wheel||py2|
|pulsar-app-0.5.0.tar.gz (172.0 kB) Copy SHA256 hash SHA256||Source||None|