hyper-shell

⚠️⚠️⚠️**Warning** ⚠️⚠️⚠️

The ``hyper-shell`` PyPI package is deprecated use ``hypershell`` instead.

For backwards compatibility the hyper-shell PyPI package will remain inplace; however, new releases will not be published there.

HyperShell is an elegant, cross-platform, high-throughput computing utility for processing shell commands over a distributed, asynchronous queue. It is a highly scalable workflow automation tool for many-task scenarios.

Built on Python and tested on Linux, macOS, and Windows.

Several tools offer similar functionality but not all together in a single tool with the user ergonomics we provide. Novel design elements include but are not limited to

Cross-platform: run on any platform where Python runs. In fact, the server and client can run on different platforms in the same cluster.
Client-server: workloads do not need to be monolithic. Run the server as a stand-alone service with SQLite or Postgres as a persistent database and dynamically scale clients as needed.
Staggered launch: At the largest scales (1000s of nodes, 100k+ of workers), the launch process can be challenging. Come up gradually to balance the workload.
Database in-the-loop: run in-memory for quick, ad-hoc workloads. Otherwise, include a database for persistence, recovery when restarting, and search.

Documentation

Documentation is available at hyper-shell.readthedocs.io. For basic usage information on the command line use: hyper-shell --help. For a more comprehensive usage guide on the command line you can view the manual page with man hyper-shell.

Contributions

Contributions are welcome. If you find bugs or have questions, open an Issue here. We’ve added a Code of Conduct recently, adapted from the Contributor Covenant, version 2.0.

Citation

If HyperShell has helped in your research please consider citing us.

@inproceedings{lentner_2022,
    author = {Lentner, Geoffrey and Gorenstein, Lev},
    title = {HyperShell v2: Distributed Task Execution for HPC},
    year = {2022},
    isbn = {9781450391610},
    publisher = {Association for Computing Machinery},
    url = {https://doi.org/10.1145/3491418.3535138},
    doi = {10.1145/3491418.3535138},
    booktitle = {Practice and Experience in Advanced Research Computing},
    articleno = {80},
    numpages = {3},
    series = {PEARC '22}
}

Metadata

Release files for hyper-shell 2.5.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for hyper-shell 2.5.1
File	Interpreter	ABI	Platform	Reset
hyper_shell-2.5.1-2-py3-none-any.whl 92.3 kB May 26, 2024	Python 3	none	any	Details

Release files / hyper_shell-2.5.1-2-py3-none-any.whl

Download URL	hyper_shell-2.5.1-2-py3-none-any.whl
Size	92.3 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`ea7ff9dcffd49e6be2d0608ebdba99513a155428fb324390e5618bd223d8cfb6`
BLAKE2b-256 checksum How to use checksums	`31377b57a7c9b6527f2b56354a5a0ad21833cfc400c346d7849b4e2e87dae5af`
Upload date	May 26, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.0 CPython/3.12.3`