Skip to main content

A tool for launching and running commands on a cluster of EC2 instances

Project description

ec2-cluster

Simple library and CLI to manage and work with clusters of EC2 instances. Multi-purpose, but created to make distributed deep learning infrastructure easier.

Goals

  • Provide the minimal set of features to run distributed deep learning training jobs on EC2 instances.
  • Provide libraries, not a framework or platform.
  • Make cluster environments reproducible to allow for parallelization of experiments
  • Make cluster launches fast
  • Focus on iterative, not disruptive, improvements on the common methodology of manually launching EC2 instances, ssh-ing to them, configuring environments by hand and running training scripts*

Quick Start

Libraries

EC2Node and EC2NodeCluster are classes for working with EC2 instances.

RemoteShell and ClusterShell

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ec2_cluster-0.0.0a2.tar.gz (11.9 kB view hashes)

Uploaded Source

Built Distribution

ec2_cluster-0.0.0a2-py3-none-any.whl (19.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page