Deploy and manage virtualized clusters for High Performance Computing (HPC) and machine learning environments on VMware software stacks
Project description
vHPC Toolkit
Virtualized High Performance Computing Toolkit (vHPC Toolkit) is a flexible, extensible and easy-to-use toolkit, that allows users to deploy and manage virtualized clusters for High Performance Computing (HPC) and machine learning environments.
This toolkit is intended to facilitate managing the lifecycle of these special configurations by leveraging vSphere APIs. It also includes features that help vSphere administrators perform some common vSphere tasks that are related to creating such high-performing environments, such as VM cloning, setting Latency Sensitivity, sizing vCPUs and memory, creating SRIOV dirtributed virtual switch (DVS) network, using assignable hardware (AH) accelerators (RDMA interconnects, GPU and FPGA), etc.
Try It Out
Prerequisites
OS for using this toolkit: Linux or Mac
vSphere >=6.5
Python >=3.7
Install
Package dependencies will be isolated in a Python virtual environment by
virtualenv
. This
allows us to have different versions of dependencies.
To install virtualenv
if you don't have one, for example,
- For Mac:
pip install --upgrade virtualenv
- For Ubuntu:
sudo apt-get install python3-pip
;sudo pip3 install virtualenv
Setup this toolkit with an virtual env:
git clone https://github.com/vmware/vhpc-toolkit.git
cd vhpc-toolkit
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
pip install -e .
# Modify config/vCenter.conf
cd bin
./vhpc_toolkit view
Follow this page about how to securely store your vSphere Client username and password in the vCenter.conf
in order to enable the possibility of automation (without prompting password every operation).
Then you should be able to execute ./vhpc_toolkit view
under vhpc_toolkit/bin
folder to view the cluster.
Documentation
Full documentation is available, or run ./vhpc_toolkit --help
under vhpc_toolkit/bin
folder to view all available
operations
Tutorial
We have a VMUG tutorial video. It covers basic to advanced usage to deploy a vHPC environment. More tutorial video will come soon.
Feel free to use these materials to teach users at your organization about vHPC toolkit.
Community
vHPC Toolkit is an open source project. Questions, discussion, and contributions are welcome. Contributions can be anything from new packages to bugfixes, documentation, or even new core features.
Resources:
- Mailing list: vhpc-toolkit@vmware.com
Contributing
The vhpc-toolkit project team welcomes contributions from the community. Before you start working with vhpc-toolkit, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.
License
This toolkit is available under the Apache 2 license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for vhpc_toolkit-1.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb56a234c2a7f2550a7055b2370fa2db2c57eeba93dc64bb3019e0f5afc9b597 |
|
MD5 | 8daa87dbd4eb41ece01f088d44f0bd26 |
|
BLAKE2b-256 | b46b24220e8bb1e5da91ea080960a41f048c0bf5afe49790eb2d85e6d09d72e9 |