Skip to main content

Command line tool for managing single-purpose cloud VMs

Project description

cloudman

Command line tool for managing single-purpose cloud VMs. Currently works with GCP.

cloudman-start-demo

Features:

  1. Single command to provision cloud-based disk images with Ubuntu, NVIDIA drivers, Anaconda, PyTorch, FastAI and more..

  2. Single command to attach a disk image, launch a cloud VM instance with any configuration of CPUs, RAM and GPU, and start Jupyter notebook.

Installation

Install the cloudman package using pip.

pip install cloudman

cloudman requires the gcloud command line tool to be installed. Make sure to complete these steps before moving forward.

  1. Install gcloud CLI using this link: https://cloud.google.com/sdk/docs/downloads-interactive

  2. Make sure the gcloud command is added to your PATH.

  3. Connect the gcloud tool your GCP account by running:

    gcloud init

  4. (Optional) Create a GCP project by running:

    gcloud projects create PROJECT_ID

Replace PROJECT_ID with a unique name e.g. kitten-puppies-999 You can also use an existing project.

  1. Activate a project by running:

    gcloud config set project PROJECT_ID

  2. Open the cloud console by running:

    cloudman console

  3. Enable billing to start creating cloud VMs.

Usage

cloudman allows you to crate standalone boot disks which can be flexibly attached to any type of machine & GPU configuration.

Create a boot disk

Running the cloudman create will create a boot disk of the given size, then install Ubuntu, NVIDIA CUDA libraries, Anaconda, Pytorch and FastAI deep learning libraries. By default, a 50 GB disk is created.

Here are some examples:

# Using default disk size of 50GB
cloudman create freesound-2019
# Custom disk size
cloudman create freesound-2019 --disk=100GB

Note that cloudman creates a temporary VM instance to install all the required packages, and then deletes the VM, but retains the disk. It uses this shell script for setup.

Start an instance

You can launch an instance with the boot disk attached using the cloudman start command. You can choose any combination of GPUs and machine types. You can also launch a CPU-only instance, which is the default. Also, you can choose whether you want a preemptible instance (enabled by default, to save cost), or a dedicated instance.

Here are some examples:

# CPU-only, preemptible
cloudman start jigsaw-ulmfit

# Same as the above command, but explicit
cloudman start jigsaw-ulmfit --gpu=nogpu --machine=auto --spot

# K80 GPU, dedicated
cloudman start jigsaw-ulmfit --gpu=k80 --nospot

Valid GPU types are: 'nogpu', 't4', 'v100', 'p100', 'p4', 'k80' Valid machine types can be found here: https://cloud.google.com/compute/docs/machine-types If machine type is set to 'auto', the tool automatically picks a reasonable machine type to match the GPU's RAM & cores. You can see the mapping here.

Once started, you can use the link to access the Jupyter server. Sometimes it may take a minute or two for the Jupyter server to start.

SSH into running instance

To SSH into a running instance use:

cloudman ssh jigsaw-ulmfit

Stop an instance

Use the cloudman stop command to stop a running instance. Example:

cloudman stop jigsaw-ulmfit

List disks and running instances

To see the available boot disks and running instances, run

cloudman list

Delete a boot disk

To delete a boot disk, run

cloudman delete jigsaw-ulmfit

This will stop any running instances and delte the boot disk complete. All your data will be lost, so be careful while using this.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cloudman-0.0.10.tar.gz (10.2 kB view details)

Uploaded Source

Built Distribution

cloudman-0.0.10-py2-none-any.whl (13.0 kB view details)

Uploaded Python 2

File details

Details for the file cloudman-0.0.10.tar.gz.

File metadata

  • Download URL: cloudman-0.0.10.tar.gz
  • Upload date:
  • Size: 10.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/39.1.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/2.7.15

File hashes

Hashes for cloudman-0.0.10.tar.gz
Algorithm Hash digest
SHA256 b290b44d1f5e23c687661ec5ad8f448950e9f960a8982932c0bfc5bd6925d38e
MD5 579cc07a9c62629458c4ea83f58f424b
BLAKE2b-256 10549308683c72e528406e37c63f230a1852512909f46e0c64f93682234a15a9

See more details on using hashes here.

File details

Details for the file cloudman-0.0.10-py2-none-any.whl.

File metadata

  • Download URL: cloudman-0.0.10-py2-none-any.whl
  • Upload date:
  • Size: 13.0 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.0 setuptools/39.1.0 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/2.7.15

File hashes

Hashes for cloudman-0.0.10-py2-none-any.whl
Algorithm Hash digest
SHA256 b3876bca29e9e945eb5ef1e1f3198ac4b988573affd1d6da2593f468e565022e
MD5 3c86772eee77d5f50c33df3f965ba8fc
BLAKE2b-256 d9b12d283ec5faa27d91f9f31cab54fc884722da23f543ff86e17f2af34aeb89

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page