Skip to main content
Pulsar Logo
Documentation Status Pulsar on the Python Package Index (PyPI) Build Status Test Coverage

This project is a Python server application that allows a Galaxy server to run jobs on remote systems (including Windows) without requiring a shared mounted file system. Unlike traditional Galaxy job runners - input files, scripts, and config files may be transferred to the remote system, the job is executed, and the results are transferred back to the Galaxy server - eliminating the need for a shared file system.

Full documentation for the project can be found on Read The Docs.

Configuring Galaxy

Galaxy job runners are configured in Galaxy’s job_conf.xml file. Some small examples of how to configure this can be found here, but be sure to check out job_conf.xml.sample_advanced in your Galaxy code base or on Github for complete information.

Quickstart

Full details on different ways to install Pulsar can be found in the install section of the documentation, but if your machine has the proper Python dependencies available, it can be quickly downloaded and a test job run with:

$ mkdir pulsar
$ cd pulsar
$ python3 -m venv venv
$ . venv/bin/activate    # venv\Scripts\activate.bat on Windows
$ pip install 'pulsar-app[web]'
$ pulsar-config
$ pulsar --daemon        # just `pulsar` on Windows
$ pulsar-check           # runs a test job

Please note that as of the 0.14.0 release, Pulsar no longer supports any version of Python 2. The minimum supported Python version is 3.5.

The configuration documentation has many details on securing your Pulsar server and enabling advanced features such as cluster integration and message queue communication.

Development and Testing

The recommended approach to setting up a development environment for Pulsar on Linux or macOS is roughly as follows:

$ git clone https://github.com/galaxyproject/pulsar
$ cd pulsar
$ python3 -m venv .venv
$ . .venv/bin/activate    # .venv\Scripts\activate on Windows
$ pip install -e '.[web]'
$ pip install -r dev-requirements.txt

The -e flag to pip installs Pulsar in “editable” mode, meaning that changes you make to the source code will be reflected when running the pulsar commands installed in the virtual environment.

This project is distributed with unit and integration tests (many of which will not run under Windows). The following command will install the needed python components to run these tests. The following command will then run these tests:

$ make tests

The following command will then produce a coverage report corresponding to this test and place it in the coverage_html_report subdirectory of this project.:

$ coverage html

Check out the Contributing documentation for many more details on developing and contributing to Pulsar.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project, you agree to abide by its terms.

Support

This documentation is an incomplete work in progress. There are more ways to configure and use Pulsar than are documented, and a growing number of Pulsar experts who would be more than happy to answer your questions and help with any problems you may run into while setting up a Pulsar deployment. Please do not hesitate to reach out on the Galaxy Admins Gitter Channel

History

0.16.0 (2026-10-11)

  • Keep client and staging package initializers lightweight, and separate Google Cloud Batch, Kubernetes, and TES clients and configuration into backend modules. COMMAND_VERSION_FILENAME remains in pulsar.client.staging and can be imported without client dependencies. Import client and staging APIs from their implementation modules; the package-level re-exports are removed. Use pulsar.client.coexecution_manager.build_client_manager for a factory supporting both ordinary clients and container coexecution. Staging objects pickled with the old package-level paths are no longer supported. Pull Request 537

0.15.16 (2026-10-08)

  • Send tool stdout and stderr to Galaxy while the job is running, enabled with send_stdout_update and tuned with stdout_update_interval, stdout_update_timeout and stdout_update_chunk_size (thanks to @gecage952). Pull Request 503

  • Recover jobs that were postprocessing when Pulsar restarted instead of losing them. Issue 354, Issue 393, Pull Request 519

  • Support configuring the AMQP heartbeat interval with amqp_heartbeat, or disabling heartbeats with false or 0 (thanks to @natefoo). Pull Request 502

  • Allow a user_mapping_script for the queued_external_drmaa manager, mapping the client-supplied user name to a local user when submitting as the real user (thanks to @bernt-matthias). Pull Request 451

  • Add native cvmfsexec support to Pulsar managers (mountrepo and namespace modes), configured via a cvmfsexec manager option or a per-job override, for accessing CVMFS repositories (and CVMFS-hosted container images) on hosts without a system-wide /cvmfs (thanks to @natefoo). Pull Request 475

  • Add a dedicated container file-action path type for rewriting resolved container image paths (e.g. a Singularity/Apptainer image on CVMFS) via a rewrite action, without overloading the unstructured tool-parameter path type. Note: path_types: "*any*" now also matches container image paths. Pull Request 475

  • Size GCP Batch jobs from the requested cores and mem, selecting a machine type when machine_type is not configured (thanks to @ksuderman). Pull Request 493

  • Support a custom GCP Batch VM boot disk image (custom_vm_image) and size (boot_disk_size_gb) (thanks to @ksuderman). Pull Request 472

  • Report DRM-side job failures as failed instead of complete (thanks to @gkr0110), and make failed terminal in StatefulManagerProxy so such jobs are deactivated, staged back, and reported to the client. Pull Request 485

  • Fail the job instead of reporting success with missing outputs when an infrastructure error (disk full, I/O error, transport error) blocks output collection or working directory stage out (thanks to @ksuderman). Pull Request 467, Pull Request 505

  • Fix a job recovery startup race where the monitor could poll recovered jobs before their external IDs were restored, without losing the lost status notification when recovery fails (thanks to @mvdbeek). Pull Request 496

  • Follow redirects in the curl transport, restart downloads the server cannot resume, and keep TUS uploads when Pulsar rebuilds a remote_transfer_tus action. Note: remote_transfer_tus now really uses TUS, so servers without tuspy fail stage out instead of silently falling back to a POST (thanks to @nuwang). Pull Request 526

  • Send MessageJobClient.get_status() requests to the status queue instead of setup, and publish failed statuses when setup fails before the job directory is readable. Pull Request 532

  • Cap tool stdout and stderr in status responses at 64 KiB, so a large maximum_stream_size cannot produce an oversized completion message (thanks to @mvdbeek). Pull Request 499

  • Keep the end of tool and job stdout/stderr when trimming them to maximum_stream_size and to the 64 KiB completion-status limit, instead of only the start. The end is usually where a failing tool explains why. Document maximum_stream_size in app.yml.sample. Issue 158, Pull Request 524

  • Use the task ID recorded by TES for polling and cancellation instead of the Galaxy job ID. Pull Request 491

  • Close leaked file, pycurl and requests handles (thanks to @mvdbeek). Pull Request 500

  • Report malformed env entries in job destinations clearly instead of crashing while building environment statements (thanks to @natefoo). Pull Request 504

  • pulsar-chown-working-directory runs chown without a shell and exits non-zero when it fails. Previously a failed chown went unnoticed by the queued_external_drmaa manager, which checks the script’s exit status.

  • Honor daemon-control arguments in --mode webless, preserve daemon logs in pulsar.log, and add a daemon installation extra (thanks to @gkr0110). Pull Request 494

  • Remove the __PULSAR_JOBS_DIRECTORY__ destination token, which never worked. Substitution only ever reached the job’s command line, so staged config files, metadata and tool scripts kept the literal token. The command line was broken too: Galaxy re-derives the job and tool directories from the working directory with os.path.abspath, which on a relative token path splices in Galaxy’s own working directory, so a single command mixed a correct /staging/<id>/working with a /galaxy/cwd//staging/<id> job directory and tool_files path. tool_script.sh lives in the job directory, so little could run. Set jobs_directory to the staging path configured on the Pulsar side instead. The per-job __PULSAR_JOB_DIRECTORY__ token used by rewrite file actions is unaffected. Pull Request 515

  • Remove the experimental Apache Mesos framework and executor. Apache Mesos has been retired to the Apache Attic, the mesos.native bindings the code imported were only ever distributed with a Mesos build, and nothing here has had a functional change since 2015. Pull Request 511

  • Bind Pulsar to the manager name Galaxy returns from compute-resource registration rather than to the relay sub claim. pulsar-config register-with-galaxy wrote the wrong name into app.yml, so registration reported success while jobs stayed queued (thanks to @dSizovs). Pull Request 514

  • Require that manager name: register-with-galaxy no longer falls back to the relay sub and no longer proposes a name in the registration payload (Galaxy mints its own and ignores ours). A registration response without a manager name now fails loudly instead of writing an app.yml bound to a guessed name (thanks to @mvdbeek). Pull Request 516

  • Document Galaxy and Pulsar compatibility per modality (REST, message queue, relay, coexecution) in a new Galaxy Compatibility docs page generated from docs/compatibility.yml. Pull Request 536

  • Document the release-branch process (master carries the next minor, release_0.N its point releases) and replace the old release targets with make release-check, make release, make release-branch, and make push-release. Backfill this section with every PR merged since 0.15.15. Pull Request 540, Pull Request 543

  • Document min_polling_interval in app.yml.sample (thanks to @martenson). Pull Request 379

  • Documentation fixes for running jobs as the real user, AMQP queue durability, and typos and grammar (thanks to @bernt-matthias, @mvdbeek, and @martincarrere). Pull Request 424, Pull Request 474, Pull Request 495, Pull Request 497

  • Add type annotations to the job managers (thanks to @bernt-matthias). Pull Request 427

  • Replace flake8 with ruff, check import order with isort in CI, and type check with Pyrefly alongside mypy (tox -e pyrefly), pinning both checkers in dev-requirements.txt. Pull Request 489, Pull Request 507, Pull Request 539

  • Harden GitHub workflows following zizmor recommendations and move CI to ubuntu-24.04 (thanks to @nsoranzo). Pull Request 478, Pull Request 506, Pull Request 508

  • Fix the resilience harness readiness checks, move it into pulsar.testing, and make the persistence and WSGI tests poll instead of sleeping (thanks to @nuwang). Pull Request 487, Pull Request 488, Pull Request 509, Pull Request 525, Pull Request 528, Pull Request 531, Pull Request 534

0.15.15 (2026-07-13)

0.15.14 (2025-01-20)

  • Fix install_wheel test by switching to docker and disabling outputs_to_working_directory (thanks to @mvdbeek). Pull Request 436

  • Add no-op BaseAction.write_from_path for actions that don’t need staging (thanks to @mvdbeek). Pull Request 435

0.15.13 (2026-01-12)

0.15.12 (2025-11-28)

0.15.11 (2025-09-30)

0.15.10 (2025-09-04)

0.15.9 (2025-07-17)

0.15.8 (2025-06-09)

0.15.7 (2025-03-13)

0.15.6 (2024-01-15)

0.15.5 (2023-09-15)

0.15.4 (2023-08-29)

0.15.3 (2023-07-20)

0.15.2 (2023-05-02)

0.15.1 (2023-04-13)

  • No changes, working around pypi isssue.

0.15.0 (2023-04-13)

0.14.16 (2022-10-04)

  • Fix small regression related to building URLs for client action mapping.

0.14.15 (2022-10-03)

  • Fix small regressions bugs in 0.14.14 - updating runner util code was bigger swap over than it seemed.

0.14.14 (2022-10-30)

0.14.13 (2021-12-06)

0.14.12 (2021-11-10)

  • Fixes to bring HOME and temp directory handling closer to Galaxy native runners.

  • Enable globbed from_work_dir outputs for remote metadata.

0.14.11 (2021-07-19)

0.14.10 (2021-07-17)

0.14.9 (2021-07-16)

  • Implement dynamic file sources abstraction for parsing files to transfer from galaxy.json files. Pull Request 269

  • Use tool classes to only test remote Galaxy tools. Pull Request 266

  • Run Galaxy framework tests against dev and master branches of Galaxy (thanks to @mvdbeek). Pull Request 264

0.14.8 (2021-07-14)

0.14.7 (2021-07-13)

0.14.6 (2021-05-24)

  • Fix for newer Galaxy tool profiles having isolated home directories.

0.14.5 (2021-04-15)

  • Potential fix for setting file actions via job destination parameters.

0.14.4 (2021-04-14)

  • Re-attempt release process - published wrong branch with 0.14.3.

0.14.3 (2021-04-13)

  • Allow transferring fewer files from Pulsar when using extended metadata with Galaxy.

0.14.2 (2021-02-15)

  • Fix the use of requests, limits, and walltime with coexecution pods. Pull Request 246

0.14.1 (2021-02-02)

0.14.0 (2020-09-17)

0.13.1 (2020-09-16)

  • Pinned all listed requirements. This is the final version of Pulsar to support Python 2.

0.13.0 (2019-06-25)

  • Various improvements and simplifications to Kubernetes job execution.

0.12.1 (2019-06-03)

  • Retry botched release that didn’t include all relevant commits.

0.12.0 (2019-06-03)

  • Revise Python Galaxy dependencies to use newer style Galaxy decomposition. galaxy-lib can no longer be installed in Pulsar’s environment, so you will likely need to rebuild your Pulsar virtualenv for this release. Pull Request 187

  • Add a Dockerfile for Pulsar with CVMFS (thanks to @nuwang and @afgane). Pull Request 166

  • Various small improvements to Kubernetes pod execution environment. Pull Request 190

  • Improve readme linting. Pull Request 186

  • Update example docs for Condor (thanks to @bgruening). Pull Request 189

0.11.0 (2019-05-16)

  • Implement staging Galaxy metadata input files in the client. 39de377

  • Fix ‘amqp_ack_republish_time’ in sample (thanks to @dannon). Pull Request 185

  • Updated amqp_url in job_conf_sample_mq_rsync.xml (thanks to @AndreasSko). Pull Request 184

  • Use wildcard char for pulsar version (thanks to @VJalili). Pull Request 181

  • Refactor toward more structured inputs. f477bc4

  • Refactor toward passing objectstore identifying information around. Pull Request 180

  • Rework imports for new Galaxy library structure. da086c9

  • Revert empty input testing, it really probably should cause a failure to transfer a non-existent file. 8bd5511

  • Better client mapper documentation. b6278b4

0.10.0 (2019-05-06)

  • Implement support for Kubernetes two container pod jobs - staging and tool execution as separate containers in the same job’s pod. Pull Request 176, Pull Request 178

0.9.1 (2019-05-01)

0.9.0 (2019-04-12)

  • Add configuration parameter to limit stream size read from disk. Pull Request 157

  • Pass full job status for failed and lost jobs. Pull Request 159

  • Improve message handling if problems occur during job setup/staging. Pull Request 160

  • Rework preprocessing job state to improve restartability and reduce job loss. This change should be applied while no jobs are running. Pull Request 164

  • Add support for overriding config through environment variables (thanks to @nuwang). Pull Request 165

  • Minor docs updates (thanks to @afgane). Pull Request 170

  • Python 3 fixes in Pulsar client (thanks to @mvdbeek). Pull Request 172

0.8.3 (2018-02-08)

  • Create universal wheels to enable Python 3 support when installing from PyPI (thanks to @nsoranzo). Pull Request 156

0.8.1 (2018-02-08)

0.8.0 (2017-09-21)

  • Support new features in Galaxy job running/scripting so that Pulsar respects $GALAXY_VIRTUAL_ENV and $PRESERVE_GALAXY_ENVIRONMENT. Fix remote metadata in cases where the tool environment changes the python on $PATH. Pull Request 137

  • Precreate Galaxy tool outputs on the remote before executing (fixes a bug related to missing output files on stage out). Pull Request 141

  • Support the remote_transfer file action without setting the jobs_directory destination param Pull Request 136

  • Fix invalid character in job managers documentation (thanks to @mapa17). Pull Request 130

  • Fix conda_auto_* option resolution and include a sample dependency_resolvers_conf.xml (thanks to @mapa17). Pull Request 132

  • Fix tox/Travis tests. Pull Request 138, Pull Request 139, Pull Request 140

  • Fix a bug with AMQP acknowledgement. Pull Request 143

0.7.4 (2017-02-07)

0.7.3 (2016-10-31)

0.7.2 (2016-08-31)

  • Fix bug causing loops on in response to preprocessing error conditions.

0.7.1 (2016-08-29)

  • Do a release to circumvent a tool version logic error in Galaxy ( released Galaxy versions think 0.7.0 < 0.7.0.dev3).

0.7.0 (2016-08-26)

  • Update Makefile to allow release pulsar as an application and a library for Galaxy at the same time.

  • Small update to test scripts for TravisCI changes.

  • Improvements for embedded Galaxy runner. (TODO: fill this out)

  • Remove support for Python 2.6. 60bf962

  • Update docs to describe project goverance and reuse Galaxy’s Code of Conduct. 7e23d43, dc47140

  • Updated cluster slots detection for SLURM from Galaxy. cadfc5a

  • Various changes to allow usage within Galaxy as a library. ce9d4f9

  • Various changes to allow embedded Pulsar managers within Galaxy. ce9d4f9, d262323, 8f7c04a

  • Introduce a separate working and metadata directory as required for Galaxy 16.04 that requires this separation. 6f4328e

  • Improve logging and comments. 38953f3, a985107, ad33cb9

  • Add Tox target for Python 2.7 unit testing. d7c524e

  • Add Makefile command for setup.py develop. fd82d00

0.6.1 (2015-12-23)

  • Tweak release process that left 0.6.0 with an incorrect PyPI description page.

0.6.0 (2015-12-23)

  • Pulsar now depends on the new galaxy-lib Python package instead of manually synchronizing Python files across Pulsar and Galaxy.

  • Numerous build and testing improvements.

  • Fixed a documentation bug in the code (thanks to @erasche). e8814ae

  • Remove galaxy.eggs stuff from Pulsar client (thanks to @natefoo). 00197f2

  • Add new logo to README (thanks to @martenson). abbba40

  • Implement an optional awknowledgement system on top of the message queue system (thanks to @natefoo). Pull Request 82 431088c

  • Documentation fixes thanks to @remimarenco. Pull Request 78, Pull Request 80

  • Fix project script bug introduced this cycle (thanks to @nsoranzo). 140a069

  • Fix config.py on Windows (thanks to @ssorgatem). Pull Request 84

  • Add a job manager for XSEDE jobs (thanks to @natefoo). 1017bc5

  • Fix pip dependency installation (thanks to @afgane) Pull Request 73

0.5.0 (2015-05-08)

  • Allow cURL downloader to resume transfers during staging in (thanks to @natefoo). 0c61bd9

  • Fix to cURL downloaders status code handling (thanks to @natefoo). 86f95ce

  • Fix non-wheel installs from PyPI. Issue 72

  • Fix mesos imports for newer versions of mesos (thanks to @kellrott). fe3e919

  • More, better logging. 2b3942d, fa2b6dc

0.4.0 (2015-04-20)

  • Python 3 support. Pull Request 62

  • Fix bug encountered when running pulsar-main and pulsar-config commands as scripts. 9d43ae0

  • Add pulsar-run script for issues commands against a Pulsar server (experimental). 3cc7f74

0.3.0 (2015-04-12)

  • Changed the name of project to Pulsar, moved to Github.

  • New RESTful web services interface.

  • SCP and Rsync file staging options added by E. Rasche. Pull Request

  • Allow YAML based configuration.

  • Support for more traditional pip/setup.py-style installs.

  • Dozens of smaller bugfixes and documentation updates.

0.2.0

  • Last version named the LWR - found on BitBucket.

  • Still supported in Galaxy as of 15.03 the release.

  • Introduced support for submitting to various queueing systems, operation as a Mesos framework, Docker support, and various other advanced deployment options.

  • Message queue support.

  • Framework for configurable file actions introduced.

0.1.0

0.0.1

Metadata

Release files for pulsar-app 0.16.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pulsar-app 0.16.0
File Size Uploaded
pulsar_app-0.16.0.tar.gz 251.7 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for pulsar-app 0.16.0
File Interpreter ABI Platform
pulsar_app-0.16.0-py3-none-any.whl Python 3 none any Details

Total release size: 516.7 kB

Release files / pulsar_app-0.16.0.tar.gz

Download URL pulsar_app-0.16.0.tar.gz
Size 251.7 kB
Tags Source
SHA-256 checksum
How to use checksums
5aa177f16d7a3fe0806bb49fe2901b4c8d1a8037fa75979d320cac4851e26ae5
BLAKE2b-256 checksum
How to use checksums
cb6b85ff2aef777641f6d5d5ee8a0c9d9e2c4b3da497d9d1a1d822106ba3ef7a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 11, 2026.

Transparency log

Release files / pulsar_app-0.16.0-py3-none-any.whl

Download URL pulsar_app-0.16.0-py3-none-any.whl
Size 265.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6189db3b220eac2426405d8713ef14433619739b793138c18be768ac5168eccb
BLAKE2b-256 checksum
How to use checksums
d79d46088168c1aed52b75995cacefd079143e3ae7e35a66b9258cd39099419c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 11, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

0.16.0 This release

2 release files

0.15.9

2 release files

0.15.7

2 release files

0.15.6

2 release files

0.15.5

2 release files

0.15.4

2 release files

0.15.3

2 release files

0.15.1

2 release files

0.14.9

2 release files

0.14.8

2 release files

0.14.7

2 release files

0.14.6

2 release files

0.14.5

2 release files

0.14.4

2 release files

0.14.3

2 release files

0.14.2

2 release files

0.14.0

2 release files

0.13.1

2 release files

0.13.0

2 release files

0.11.0

2 release files

0.9.1

2 release files

0.9.0

2 release files

0.8.3

2 release files

0.8.2

2 release files

0.8.1

2 release files

0.8.0

2 release files

0.7.4

3 release files

0.7.3

3 release files

0.7.2

3 release files

0.7.1

3 release files

0.7.0

3 release files

0.6.1

3 release files

0.5.0

3 release files

0.4.0

3 release files

0.3.0

3 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page