vhpc-toolkit 1.0.3

Deploy and manage virtualized clusters for High Performance Computing (HPC) and machine learning environments on VMware software stacks

vHPC Toolkit

Virtualized High Performance Computing Toolkit (vHPC Toolkit) is a flexible, extensible and easy-to-use toolkit, that allows users to deploy and manage virtualized clusters for High Performance Computing (HPC) and machine learning environments.

This toolkit is intended to facilitate managing the lifecycle of these special configurations by leveraging vSphere APIs. It also includes features that help vSphere administrators perform some common vSphere tasks that are related to creating such high-performing environments, such as VM cloning, setting Latency Sensitivity, sizing vCPUs and memory, creating SRIOV dirtributed virtual switch (DVS) network, using assignable hardware (AH) accelerators (RDMA interconnects, GPU and FPGA), etc.

Try It Out

Prerequisites

OS for using this toolkit: Linux or Mac
vSphere >=6.5
Python >=3.7

Install

Package dependencies will be isolated in a Python virtual environment by virtualenv. This allows us to have different versions of dependencies.

To install virtualenv if you don't have one, for example,

• For Mac: pip install --upgrade virtualenv
• For Ubuntu: sudo apt-get install python3-pip; sudo pip3 install virtualenv

Setup this toolkit with an virtual env:

git clone https://github.com/vmware/vhpc-toolkit.git
cd vhpc-toolkit
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
pip install -e .
# Modify config/vCenter.conf
cd bin
./vhpc_toolkit view

Follow this page about how to securely store your vSphere Client username and password in the vCenter.conf in order to enable the possibility of automation (without prompting password every operation).

Then you should be able to execute ./vhpc_toolkit view under vhpc_toolkit/bin folder to view the cluster.

Documentation

Full documentation is available, or run ./vhpc_toolkit --help under vhpc_toolkit/bin folder to view all available operations

Tutorial

We have a VMUG tutorial video. It covers basic to advanced usage to deploy a vHPC environment. More tutorial video will come soon.

Feel free to use these materials to teach users at your organization about vHPC toolkit.

Community

vHPC Toolkit is an open source project. Questions, discussion, and contributions are welcome. Contributions can be anything from new packages to bugfixes, documentation, or even new core features.

Resources:

• Mailing list: [e-mail]

Contributing

The vhpc-toolkit project team welcomes contributions from the community. Before you start working with vhpc-toolkit, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.

License

This toolkit is available under the Apache 2 license.