Use AWS CDK to create a gitlab runner, and use gitlab runner to help you execute your Gitlab pipeline job.
Project description
Welcome to cdk-gitlab-runner
Use AWS CDK to create gitlab runner, and use gitlab runner to help you execute your Gitlab Pipeline Job.
GitLab Runner is the open source project that is used to run your CI/CD jobs and send the results back to GitLab. (source repo)
Why
Gitlab provides 400 minutes per month for each free user, hosted Gitlab Runner to execute your gitlab pipeline job.That's pretty good and users don't need to manage gitlab runner. If it is just a simple ci job for test 400, it may be enough. But what if you want to deploy to your AWS production environment through pipeline job? Is there any security consideration for using the hosted gitlab runner?!
But creating Gitlab Runner is not that simple, so I created this OSS so that you can quickly create Gitlab Runner and delete your Gitlab Runner via AWS CDK. It will be used with AWS IAM Role, so you don't need to put AKSK in Gitlab environment variables.
Note
Default will help you generate below services:
-
VPC
- Public Subnet (2)
-
EC2 (1 T3.micro)
Before start you need gitlab runner token in your gitlab project
or gitlab group
In Group
Group > Settings > CI/CD
In Project
Project > Settings > CI/CD > Runners
Usage
Replace your gitlab runner token in $GITLABTOKEN
Instance Type
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner
# If want change instance type to t3.large .
GitlabContainerRunner(self, "runner-instance", gitlabtoken="$GITLABTOKEN", ec2type="t3.large")
# OR
# Just create a gitlab runner , by default instance type is t3.micro .
from cdk_gitlab_runner import GitlabContainerRunner
GitlabContainerRunner(self, "runner-instance", gitlabtoken="$GITLABTOKEN")
Gitlab Server Customize Url .
If you want change what you want tag name .
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
# If you want change what your self Gitlab Server Url .
from cdk_gitlab_runner import GitlabContainerRunner
GitlabContainerRunner(self, "runner-instance-change-tag",
gitlabtoken="$GITLABTOKEN",
gitlaburl="https://gitlab.my.com/"
)
Tags
If you want change what you want tag name .
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
# If you want change what you want tag name .
from cdk_gitlab_runner import GitlabContainerRunner
GitlabContainerRunner(self, "runner-instance-change-tag",
gitlabtoken="$GITLABTOKEN",
tags=["aa", "bb", "cc"]
)
IAM Policy
If you want add runner other IAM Policy like s3-readonly-access.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
# If you want add runner other IAM Policy like s3-readonly-access.
from cdk_gitlab_runner import GitlabContainerRunner
from aws_cdk.aws_iam import ManagedPolicy
runner = GitlabContainerRunner(self, "runner-instance-add-policy",
gitlabtoken="$GITLABTOKEN",
tags=["aa", "bb", "cc"]
)
runner.runner_role.add_managed_policy(
ManagedPolicy.from_aws_managed_policy_name("AmazonS3ReadOnlyAccess"))
Security Group
If you want add runner other SG Ingress .
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
# If you want add runner other SG Ingress .
from cdk_gitlab_runner import GitlabContainerRunner
from aws_cdk.aws_ec2 import Port, Peer
runner = GitlabContainerRunner(self, "runner-add-SG-ingress",
gitlabtoken="GITLABTOKEN",
tags=["aa", "bb", "cc"]
)
# you can add ingress in your runner SG .
runner.default_runner_sG.connections.allow_from(
Peer.ipv4("0.0.0.0/0"),
Port.tcp(80))
Use self VPC
2020/06/27 , you can use your self exist VPC or new VPC , but please check your
vpc public Subnet
Auto-assign public IPv4 address must be Yes ,orvpc private Subnet
route table associatednat gateway
.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner
from aws_cdk.aws_ec2 import Port, Peer, Vpc, SubnetType
from aws_cdk.aws_iam import ManagedPolicy
newvpc = Vpc(stack, "VPC",
cidr="10.1.0.0/16",
max_azs=2,
subnet_configuration=[SubnetConfiguration(
cidr_mask=26,
name="RunnerVPC",
subnet_type=SubnetType.PUBLIC
)
],
nat_gateways=0
)
runner = GitlabContainerRunner(self, "testing",
gitlabtoken="$GITLABTOKEN",
ec2type="t3.small",
selfvpc=newvpc
)
Use your self exist role
2020/06/27 , you can use your self exist role assign to runner
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner
from aws_cdk.aws_ec2 import Port, Peer
from aws_cdk.aws_iam import ManagedPolicy, Role, ServicePrincipal
role = Role(self, "runner-role",
assumed_by=ServicePrincipal("ec2.amazonaws.com"),
description="For Gitlab EC2 Runner Test Role",
role_name="TestRole"
)
runner = GitlabContainerRunner(stack, "testing",
gitlabtoken="$GITLAB_TOKEN",
ec2iamrole=role
)
runner.runner_role.add_managed_policy(
ManagedPolicy.from_aws_managed_policy_name("AmazonS3ReadOnlyAccess"))
Custom Gitlab Runner EBS szie
2020/08/22 , you can change you want ebs size.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner
GitlabContainerRunner(stack, "testing",
gitlabtoken="$GITLAB_TOKEN",
ebs_size=50
)
Control the number of runners with AutoScalingGroup
2020/11/25 , you can set the number of runners.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabRunnerAutoscaling
GitlabRunnerAutoscaling(stack, "testing",
gitlab_token="$GITLAB_TOKEN",
min_capacity=2,
max_capacity=2
)
Support Spotfleet Gitlab Runner
2020/08/27 , you can use spotfleet instance be your gitlab runner, after create spotfleet instance will auto output instance id.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner, BlockDuration
runner = GitlabContainerRunner(stack, "testing",
gitlabtoken="GITLAB_TOKEN",
ec2type="t3.large",
block_duration=BlockDuration.ONE_HOUR,
spot_fleet=True
)
# configure the expiration after 1 hours
runner.expire_after(Duration.hours(1))
2020/11/19, you setting job runtime bind host volumes. see more https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
from cdk_gitlab_runner import GitlabContainerRunner, BlockDuration
runner = GitlabContainerRunner(stack, "testing",
gitlabtoken="GITLAB_TOKEN",
ec2type="t3.large",
docker_volumes=[{
"host_path": "/tmp/cahce",
"container_path": "/tmp/cahce"
}
]
)
Wait about 6 mins , If success you will see your runner in that page .
you can use tag gitlab
, runner
, awscdk
,
Example gitlab-ci.yaml
dockerjob:
image: docker:18.09-dind
variables:
tags:
- runner
- awscdk
- gitlab
variables:
DOCKER_TLS_CERTDIR: ""
before_script:
- docker info
script:
- docker info;
- echo 'test 123';
- echo 'hello world 1228'
If your want to debug you can go to aws console
In your runner region !!!
AWS Systems Manager > Session Manager > Start a session
click your runner
and click start session
in the brower console in put bash
# become to root
sudo -i
# list runner container .
root# docker ps -a
# modify gitlab-runner/config.toml
root# cd /home/ec2-user/.gitlab-runner/ && ls
config.toml
:clap: Supporters
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cdk-gitlab-runner-1.115.57.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7705aebb62bb62292034625bc53568a278643b15bd734c628742c1af4423bed6 |
|
MD5 | 5e4ae0d7cb8911fd68cca0f9a3a7005e |
|
BLAKE2b-256 | 171f72c64f765be1a6d176df5064624d96edca90a4f5c907472455550502281d |
Hashes for cdk_gitlab_runner-1.115.57-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fbbb5dbc472c0dc8af0e0f206d730dc1512fc5e1c1e8574795091a7d3d80ffc |
|
MD5 | cf647f7a4603e4d364c049fcb97556de |
|
BLAKE2b-256 | dc7da8da3b3c5dadf795ec7af7d0c810b605363d233a77a7d3dcd55adb7b79fe |