AWS ECS Deployment Tool With Terraform
Project description
Introduce
ECS deploy using docker compose
and terraform
.
You need to manage just yml
file for docker compose
and:
ecsdep cluster create
ecsdep service up
That's all.
Running Docker
Local Testing
Docker contains terrform
, awscli
and ecsdep
.
docker run -d --privileged \
--name docker \
-v /path/to/myproject:/app \
hansroh/dep:dind
docker exec -it docker bash
Within docker,
pip3 install -U ecsdep
Gitlab CI/CD
Add these lines into .gitlab-ci.yml
:
image: hansroh/dep:latest
services:
- name: docker:dind
alias: dind-service
before_script:
- pip3 install -U ecsdep
Creating ECS Cluster and Deploy Services
Creating Base Dcoker Compose Config File
Create Dockerfile
and docker-compose.yml
.
This example launch 1 container named skitai-app
.
version: '3.3'
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
container_name: skitai-app
build:
context: .
dockerfile: ./Dockerfile
ports:
- 5000:5000
Make sure service name (skitai-app) is same as container_name
.
And test image build and container.
docker-compose build
docker-compose up -d
docker-compose down
Adding ECS Related Settings
Specify Deploy Containers
Add services.skitai-app.deploy
key. Otherwise this container will not be included.
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
deploy:
Image Registry Pull Credential
For private registry, you need credential. See this.
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
x-ecs-pull-credentials: arn:aws:secretsmanager:ap-northeast-2:000000000:secret:gitlab/registry/mysecret-PrENMF
Logging (Optaional)
For integrating Cloud Watch log group.
services:
skitai-app:
logging:
x-ecs-driver: awslogs
Container Health Checking (Optaional)
healthcheck:
test:
- "CMD-SHELL"
- "wget -O/dev/null -q http://localhost:5000 || exit 1"
Container Level Reosurce Requirements
For minimal memory reservation - called soft memory limit,
services:
skitai-app:
deploy:
resources:
reservations:
memory: "256M"
For hard memory limit,
deploy:
resources:
reservations:
memory: "256M"
limits:
memory: "320M"
Make sure hard limits must be greater or equal than reservation value.
For miniaml CPU units reservation,
deploy:
resources:
reservations:
cpus: "1024"
1024 means 1 vCPU. ECS will deploys container which fulfills these reservation requirements.
Value of cpus
must be string.
Maybe you are using GPU,
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
But ecsdep
ignores above statement. Just add x-ecs-gpus
.
deploy:
resources:
reservations:
x-ecs-gpus: 1
devices: ...
Ok, test docker build and container.
docker-compose build
Terraform Setting
services:
skitai-app:
...
x-terraform:
provider: aws
region: ap-northeast-2
state-backend:
region: "ap-northeast-2"
bucket: "states-data"
key-prefix: "terraform/ecs-cluster"
Make sure you create s3 bucket.
Cluster Settings
Prerequisitions
-
AWS/Gitlab credentials
docker login -u <gitlab username> -p <gitlab deploy-token> registry.gitlab.com aws configure set aws_access_key_id <AWS_ECS_ACCESS_KEY_ID> aws configure set aws_secret_access_key <AWS_ECS_SECRET_ACCESS_KEY>
-
AWS certification for your service domain (ex. mydomain.com and *.mydomain.com)
-
SSH key file, for generating key file,
ssh-keygen -t rsa -b 2048 -C "email@example.com" -f ./mykeypair
It makes 2 files.
mykeypair
is private key andmykeypair.pub
is public key. Please keep private key file carefully for accessing your EC2 instances.Also
mykeypair.pub
must be placed with current yaml file.
services:
skitai-app:
...
x-ecs-cluster:
name: my-cluster
public-key-file: "mykeypair.pub"
instance-type: t3.medium
ami: amzn2-ami-ecs-hvm-*-x86_64-*
autoscaling:
desired: 2
min: 2
max: 10
cpu: 60
memory: 60
loadbalancer:
cert-name: mydoamin.com
vpc:
cidr_block: 10.0.0.0/16
octet3s: [7, 8, 9]
It creates resources like:
- Load Balancer
- VPC with subnets: 10.0.7.0/24, 10.0.8.0/24 and 10.0.9.0/24
- Cluster Instance Auto Scaling Launch Configuration
- Cluster Auto Scaling Group
- Cluster
autoscaling.cpu
mean that your cpu reservations of your containers
reaches 60% of all cluster instances's CPU units, scale out ECS instances.
If 0, nothing will be happen.
autoscaling.memory
is same.
If you specified x-ecs-gpus
, instance-type
must have GPU.
If you consider VPC peering, choose carefully cidr_block
.
octet3s
will be allocated into each availavility zones. It means octet3s
length determines number of availavility zones, so it cannot be greater than
number of availavility zones in current region.
Createing/Destroying ECS Cluster
Finally you can create cluster,
ecsdep
find default yaml named as docker-compose.yml
or compose.ecs.yml
.
ecsdep cluster create
ecsdep cluster destroy
Specifying file is also possible.
ecsdep -f /path/to/docker-compose.yml cluster create
ecsdep -f /path/to/docker-compose.yml cluster destroy
Deploying ervice
x-ecs-service:
name: ecsdep
stages:
default:
env-service-stage: "qa"
hosts: ["qa.myservice.com"]
listener-priority: 100
production:
env-service-stage: "production"
hosts: ["myservice.com"]
listener-priority: 101
autoscaling:
min: 3
max: 7
loadbalancing:
pathes:
- /*
protocol: http
healthcheck:
path: "/ping"
matcher: "200,301,302,404"
deploy:
compatibilities:
- ec2
resources:
limits:
memory: 256M
cpus: "1024"
autoscaling:
desired_count: 1
min: 1
max: 4
cpu: 100
memory: 80
Staging
stages
can make deploy stages like production
, qa
or staging
.
It find your current environment variable named SERVICE_STAGE
.
export SERVICE_STAGE=qa
Current deploy stages is selected by current SERVICE_STAGE
value matched with env-service-stage
.
If SERVICE_STAGE
is qa
, your container routed qa.myservice.com
by load balancer.
Auto Scaling
Default container-level auto scaling settings are placed in x-ecs-service.autoscaling
.
But each stages can overwrite this values by x-ecs-service.[workspace name].autoscaling
.
Load Balancing (Optional)
Your container routed to loadbalancing.pathes
by your load balancer.
Using Fargate
If you wnat fargate
deploy, set x-ecs-service.deploy.compatibilities
to - fargate
Resource Limiting
It is required when 'fargate' laubching, but it is optional for 'ec2' launching.
x-ecs-service.deploy.autoscaling.cpu
and x-ecs-service.deploy.autoscaling.memory
are both percentages of reserved cpu units or memory MBytes.
If x-ecs-service.deploy.resources.limits.cpus
is defined, service cannot use units
over this value.
If x-ecs-service.deploy.resources.limits.memory
is defined and your container over
this value, container will be terminated.
Deploying Service
# image tagged with first 8chars from git commit hash string
# it will be provided on gitlab runner, but at local testing,
# lateset is just OK
export CI_COMMIT_SHA=latest
export SERVICE_STAGE=qa
ecsdep -f dep/compose.ecs.yml service up
Whenever commanding ecsdep service up
, your containers will be deployed to ECS by rolling update way.
As a results, AWS resources will be created.
- Task Definition
- Update Service and Run
Shutdown/Remove Service
ecsdep service down
Deploying Other Services Into Cluster
It is recommended cluster settings are kept in your main app only.
In other service's yml, keep x-ecs-cluster.name
only and remove other x-ecs-cluster
settings.
You just care about x-ecs-service
and services
definition.
services:
skitai-app-2:
...
x-terrform:
...
x-ecs-cluster:
name: my-cluster
Deploying Service With Multiple Containers
services:
skitai-app:
deploy:
ports:
- 5000
healthcheck:
test:
- "CMD-SHELL"
- "wget -O/dev/null -q http://localhost:5000 || exit 1"
skitai-nginx:
depends_on:
- skitai-app
x-ecs-wait-conditions:
- HEALTHY
ports:
- "80:80"
Make sure only single loadbalancable container can have host port mapping like "80:80", other wise just docker internal port like "5000".
Deploying Non Web Service
Get rid of ports and load balancing settings like:
services.your-app.ports
x-ecs-service.loadbalancing
x-ecs-service.stages.default.hosts
x-ecs-service.stages.default.listener-priority
Using Secrets
version: '3.3'
services:
skitai-app:
environment:
DB_PASSWORD=$DB_PASSWORD
secrets:
DB_PASSWORD:
name: "arn:aws:secretsmanager:ap-northeast-1:0000000000:secret:gitlab/registry/hansroh-PrENMF:DBPASSWORD::"
external: true
At deploy time, environment DB_PASSWORD
is overwritten with
secrets.DB_PASSWORD.name
value by AWS ECS Service. see AWS Secret Manager document.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.