Flux Hierarchy
Create trees of Flux instances
🚧 under development and experimental 🚧
This tool enables generation and orchestration of Flux hierarchies, or trees of instances. Such a setup can enable programmatic organization and submission of commands, or high throughput. Use cases we want to address:
- Creation (and organization) of a Flux Hierarchy
- Discovery of an existing Flux Hierarchy (e.g, for MCP)
Usage
Let's first create a hierarchy. This will be a Flux job. You'll need to be in a Flux instance where a handle is discoverable. E.g., in the DevContainer:
flux start
Then create a simple, flat hierarchy with all the resources allocated to one broker.
flux-hierarchy start ./examples/hierarchy-one.yaml
You can test throughput (this also starts the hierarchy):
flux-hierarchy throughput ./examples/hierarchy-one.yaml
For either of the above, the hierarchy will continue running (and you need to cancel the job).
flux cancel $(flux job last)
You can also view the shape of the hierarchy without running anything:
flux-hierarchy view ./examples/hierarchy-one.yaml
$ flux-hierarchy view ./examples/corona/hierarchy-2.yaml
=>
🌿 Leaf Broker Workers...{}
level1 [Nodes: 2]
├── level2 [Nodes: 1, Cores: 48]
└── level2 [Nodes: 1, Cores: 48]
Development
To build and release:
python3 -m build
# or
python3 setup.py sdist bdist_wheel
twine upload dist/flux-hierarchy-<version>*
WIP / TODO / Would be nice
- I can't remember command to get
<host>:<rank>mapping (I came up with something) - Use kvs for uris, saving results, etc. instead of the local dir.
- Have local throughput wait for results not rely on filesystem results (use job wait)
- Some means to deploy submit to node as a service on the node (that knows about URIs)
- Save result to kvs or similar (not filesystem)
- Should be able to read in directory of active sockets to generate tree
- Allow different job shapes / specs.
- Expose simulation duration time
- Expose other resource params
- A check for non-local mode that we aren't going to ddos ssh.
License
HPCIC DevTools is distributed under the terms of the MIT license. All new contributions must be made under this license.
See LICENSE, COPYRIGHT, and NOTICE for details.
SPDX-License-Identifier: (MIT)
LLNL-CODE- 842614
Metadata
Release files for flux-hierarchy 0.0.13
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| flux_hierarchy-0.0.13.tar.gz | 23.8 kB | Without provenance | Details for flux_hierarchy-0.0.13.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| flux_hierarchy-0.0.13-py3-none-any.whl | Python 3 | none | any | Details for flux_hierarchy-0.0.13-py3-none-any.whl |
Total release size: 48.8 kB
Release files / flux_hierarchy-0.0.13.tar.gz
| Download URL | flux_hierarchy-0.0.13.tar.gz |
|---|---|
| Size | 23.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6d9e9cf5d47cb2fe48a5f2e08a2a69c8f182e26b3aa4f57f860ae50ccf788539
|
|
BLAKE2b-256 checksum How to use checksums |
53d4d516b11c7e7dd59f46a16f49a3551ec40a2dcc76f017dc77b8b220ebf322
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.12.2
|
Release files / flux_hierarchy-0.0.13-py3-none-any.whl
| Download URL | flux_hierarchy-0.0.13-py3-none-any.whl |
|---|---|
| Size | 25.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
12baff4e4a36a50b7cec58e51a6cada80a0e5c44e3b38dcd2c41bd42333c88f6
|
|
BLAKE2b-256 checksum How to use checksums |
4fb33bd50f0ef7f44e3a3a9d47a45cb94e44156c4a20c6178d85252be3e0d384
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.12.2
|
Security for flux-hierarchy 0.0.13
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.