jupyter-deploy-tf-aws-ec2-base 0.1.0a2

Base terraform template to deploy jupyter notebook on AWS EC2.

AWS EC2 instance running a Jupyter Server using a Traefik proxy

---

This terraform project creates an EC2 instance in the default VPC and route 53 records in a domain you own. Within the EC2 instance, it runs a jupyter service, a traefik service for proxy and an oauth sidecar for authentication and authorization.

The instance is configured so that you can access it using AWS SSM.

This project:

• places the instance in the first subnet of the default VPC
• select the latest AL 2023 AMI for x86_64 architecture
• sets up an IAM role to enable SSM access
• passes on the root volume of the AMI
• adds an EBS volume which will mount on the Jupyter Server container
• creates an SSM instance-startup script, which references several files:
  • cloudinit.sh.tftpl for the basic setup of the instance
  • docker-compose.yml.tftpl for the docker services to run in the instance
  • docker-startup.sh.tftpl to run the docker-compose up cmd and post docker-start instructions
  • traefik.yml.tftpl traefik configuration file
  • dockerfile.jupyter for the Jupyter container
  • jupyter-start.sh as the entrypoint script for the Jupyter container
  • jupyter-reset.sh as the fallback script if the Jupyter container fails to start
  • pyproject.jupyter.toml for Python dependencies of the base environment where the Jupyter server runs
  • jupyter_server_config.py for Jupyter server configuration
  • update_users.sh as a utility script for updating the currently authenticated users
• creates an SSM association, which runs the startup script on the instance
• creates the Route 53 Hosted Zone for the domain unless it already exists
• adds DNS records to the Route 53 Hosted Zone
• creates an AWS Secret to store the OAuth App client secret
• provides two presets default values for the template variables:
  • defaults-all.tfvars comprehensive preset with all the recommended values
  • defaults-base.tfvars more limited preset; it will prompt user to select the instance type and volume size

Prerequisites

• a domain that you own verifiable by route 53
  • instructions to register a domain
  • instructions to acquire a domain
• a GitHub OAuth App
  • instructions to create a new app
  • you'll need the app client ID and client secret
• a list of GitHub usernames to authorize

Usage

This terraform project is meant to be used with jupyter-deploy.

Installation (with pip):

Create or activate a python environment.

pip install jupyter-deploy
pip install jupyter-deploy-tf-aws-ec2-base

Project setup

Consider making my-jupyter-deployment a git repository.

mkdir my-jupyter-deployment
cd my-jupyter-deployment

jd init . -E terraform -P aws -I ec2 -T base

Configure and create the infrastructure

jd config
jd up

Access your notebook

jd open

Requirements

Name	Version
terraform	>= 1.0
aws	>= 4.66
github	~> 6.0

Providers

Name	Version
aws	>= 4.66
github	~> 6.0

Modules

No modules.

Resources

Name	Type
aws_security_group	resource
aws_instance	resource
aws_iam_role	resource
aws_iam_role_policy_attachment	resource
aws_iam_instance_profile	resource
aws_ebs_volume	resource
aws_volume_attachment	resource
aws_ssm_document	resource
aws_ssm_association	resource
aws_route53_zone	resource
aws_route53_record	resource
aws_secretsmanager_secret	resource
aws_iam_policy	resource
aws_ssm_parameter	resource
null_resource	resource
aws_default_vpc	resource
aws_subnets	data source
aws_subnet	data source
aws_ami	data source
aws_route53_zone	data source
aws_iam_policy	data source
aws_iam_policy_document	data source
local_file	data source

Inputs

Name	Type	Default	Description
region	string	us-west-2	AWS region where the resources should be created
instance_type	string	t3.medium	The type of instance to start
key_pair_name	string	null	The name of key pair
ami_id	string	null	The ID of the AMI to use for the instance
volume_size_gb	number	30	The size in GB of the EBS volume the Jupyter Server has access to
volume_type	string	gp3	The type of EBS volume the Jupyter Server will has access to
iam_role_prefix	string	Jupyter-deploy-ec2-base	The prefix for the name of the IAM role for the instance
oauth_app_secret_prefix	string	Jupyter-deploy-ec2-base	The prefix for the name of the AWS secret where to store your OAuth app client secret
letsencrypt_email	string	Required	An email for letsencrypt to notify about certificate expirations
domain	string	Required	A domain that you own
subdomain	string	Required	A sub-domain of domain to add DNS records
oauth_provider	string	github	The OAuth provider to use
oauth_allowed_usernames	list(string)	Required	The list of GitHub usernames to allowlist
oauth_app_client_id	string	Required	The client ID of the OAuth app
oauth_app_client_secret	string	Required	The client secret of the OAuth app
custom_tags	map(string)	{}	The custom tags to add to all the resources

Outputs

Name	Description
jupyter_url	The URL to access your notebook app
auth_url	The URL for the OAuth callback - do not use directly
instance_id	The ID of the EC2 instance
ami_id	The Amazon Machine Image ID used by the EC2 instance
jupyter_server_public_ip	The public IP assigned to the EC2 instance
secret_arn	The ARN of the AWS Secret storing the OAuth client secret