hypershell

HyperShell is an elegant, cross-platform, high-throughput computing utility for processing shell commands over a distributed, asynchronous queue. It is a highly scalable workflow automation tool for many-task scenarios.

Built on Python and tested on Linux, macOS, and Windows.

Other tools may offer similar functionality in some places but not within a single tool and not with the flexibility, ergonomics, and scalability provided by HyperShell.

Design elements include but are not limited to:

Client-server: Run the server in stand-alone mode with SQLite or PostgreSQL.

Scale clients elastically as needed (even down to zero).
Cross-platform: trivial to install, run on any platform where Python runs.

Mix platforms within a running cluster (Server on Linux, Clients on Windows).
Staggered launch: Come up gradually to balance the workload.

Scale to 1000+ nodes, 250k+ workers without crashing the server.
Database in-the-loop: persist task metadata across runs.

Fault-tolerant by default. Automated retries. Task history.
User-defined tags: annotate tasks with key:value tags.

Manage catalogs of large collections of tasks with ease.

Usage

HyperShell is primarily a command-line program. Most users will operate the hs cluster command (hsx for short) in a start-to-finish workflow scenario much like people tend to do with alternatives like xargs, GNU Parallel, or HPC-specific tools like ParaFly or TaskFarmer (NERSC-only) or Launcher (TACC).

seq 1000000 | hsx -t 'echo {}' -N64 --ssh 'a[00-32].cluster' > task.out

Documentation

Documentation is available at hypershell.readthedocs.io. For basic usage information on the command line use: hs --help. For a more comprehensive usage guide on the command line you can view the manual page with man hs.

Support and Contributions

Join the Discord server to post questions, discuss your project, share with the community, keep in touch with announcements and upcoming events!

HyperShell is an open-source project developed on GitHub. If you find bugs or issues with the software please create an Issue. Contributions are welcome in the form of Pull requests for bug fixes, documentation, and minor feature improvements.

We’ve added a Code of Conduct recently, adapted from the Contributor Covenant, version 2.0.

License

HyperShell is released under the Apache Software License (v2).

Citation

If HyperShell has helped in your research please consider citing us.

@inproceedings{lentner_2022,
    author = {Lentner, Geoffrey and Gorenstein, Lev},
    title = {HyperShell v2: Distributed Task Execution for HPC},
    year = {2022},
    isbn = {9781450391610},
    publisher = {Association for Computing Machinery},
    url = {https://doi.org/10.1145/3491418.3535138},
    doi = {10.1145/3491418.3535138},
    booktitle = {Practice and Experience in Advanced Research Computing},
    articleno = {80},
    numpages = {3},
    series = {PEARC '22}
}

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hypershell-2.8.1.tar.gz (3.7 MB view details)

Uploaded Jul 8, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

hypershell-2.8.1-py3-none-any.whl (236.9 kB view details)

Uploaded Jul 8, 2026 Python 3

File details

Details for the file hypershell-2.8.1.tar.gz.

File metadata

Download URL: hypershell-2.8.1.tar.gz
Upload date: Jul 8, 2026
Size: 3.7 MB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for hypershell-2.8.1.tar.gz
Algorithm	Hash digest
SHA256	`12eae300a7fa0cc0373f163fa9a200d7506733120045826352ee00c469b00b57`
MD5	`b89c14286362acc446547ae03b78c466`
BLAKE2b-256	`a50c6e6c22a025bb4baa6ae53b329000158dc87edc45b804095acff6e3cc680a`

See more details on using hashes here.

File details

Details for the file hypershell-2.8.1-py3-none-any.whl.

File metadata

Download URL: hypershell-2.8.1-py3-none-any.whl
Upload date: Jul 8, 2026
Size: 236.9 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for hypershell-2.8.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`3160d764018ee12013eb4510f548f4c00e64d861c0c0e00fbd3b3f5da11ecccf`
MD5	`4b032d0f1a5edc0f268204d06cf85b43`
BLAKE2b-256	`b6f99b085869e4f2c9df8205f6bb5748d46f8c49e9756c7f05fffa5839072dab`