Skip to main content

R clusters on AWS

Project description

rcluster makes launching and accessing an R cluster on AWS simple and accessible.

This repository will:

  • Create a connection to your AWS account
  • Create and save an AMI with the software and configuration needed
  • Allow you to launch a manager and a stated number of worker nodes, automating the network connections between them and hosting a common NFS-based home folder under the default cluster user:
    • “/home/cluster” is shared between manager and all workers
    • “/home/cluster/hostfile” contains the IPs of accessible worker nodes and spare manager nodes, repeated based on the number of available worker cores
    • cluster user’s “.Rprofile” defines the R function defaultCluster() that automatically reference the hostfile to create a PSOCK-based cluster

After that, login to RStudio Server as normal on the manager, run defaultCluster(), and use the returned parallel cluster object with parLapply() and its peers.

Getting Started

First, you must create and save locally your AWS access key ID and secret access key (instructions).

Next, run rcluster-config from your command line. Note that this function will, by default, write your AWS access key and secret access key to a hidden folder in your user directory with no additional security.

There are currently three functions to launch and manage an R cluster:

  • rcluster - Launch an R cluster on AWS using the default configuration file. This function will open your default browser to the RStudio Server login page on the manager instance once it is live.
  • rcluster-open - Access an active R cluster (opens a new tab in your web browser to the RStudio Server instance, if available).
  • rcluster-terminate - Terminate all instances associated with your rcluster configuration.

Project details


Release history Release notifications

This version
History Node

0.2.27

History Node

0.2.26

History Node

0.2.25

History Node

0.2.24

History Node

0.2.23

History Node

0.2.22

History Node

0.2.21

History Node

0.2.20

History Node

0.2.19

History Node

0.2.18

History Node

0.2.17

History Node

0.2.16

History Node

0.2.15

History Node

0.2.14

History Node

0.2.13

History Node

0.2.12

History Node

0.2.9

History Node

0.2.8

History Node

0.2.7

History Node

0.2.6

History Node

0.2.5

History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
rcluster-0.2.27-py3-none-any.whl (16.6 kB) Copy SHA256 hash SHA256 Wheel py3 Nov 21, 2016
rcluster-0.2.27.zip (17.2 kB) Copy SHA256 hash SHA256 Source None Nov 21, 2016

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page