Prometheus exporter for AWS ECS Task and Container Metrics
Project description
ecs-container-exporter
AWS ECS side car that exports ECS container level docker stats metrics to Prometheus as well as publish it via Statsd.
Motivation
The default metrics available in AWS ECS are limited, mostly at the task level, across all containers in a task; the container level metrics are not available. In addition, more detailed cgroup metrics are also not available, such as per cpu, and memory usage breakdown into cache, rss, etc.
Luckily AWS exposes the docker stats data via a Task metadata endpoint.
The ecs-container-exporter parses this data, and can expose it to Prometheus or push them via StatsD.
Usage
Install via Pip:
$ pip3 install ecs-container-exporter
or via docker:
$ docker pull raags/ecs-container-exporter
On ECS, add the following json to the task definition:
{
"name": "ecs-container-exporter",
"image": "raags/ecs-container-exporter:latest",
"portMappings": [
{
"hostPort": 0,
"protocol": "tcp",
"containerPort": 9545
}
],
"command": [],
"cpu": 256,
"dockerLabels": {
"PROMETHEUS_EXPORTER_PORT": "9545"
}
}
The PROMETHEUS_EXPORTER_PORT
label is for ECS discovery via https://github.com/teralytics/prometheus-ecs-discovery
To include or exclude application containers use the INCLUDE
or EXCLUDE
environment variable. By default ecs-container-exporter
and ~internal~ecs~pause
(a Fargate internal sidecar) is excluded.
Statsd
Version 2.0.0
add Statsd support with --use-statsd
flag or env USE_STATSD
. Metrics are emitted with DogStatsd Tag format.
Metrics
The metrics are sampled twice as per the configured interval
(default 60s), and than aggregated in this interval. This should be set to the Prometheus scrape interval.
CPU
CPU usage ratio is calculated and scaled as per the applicable container or task cpu limit:
Task Limit | Container Limit | Task Metric | Container Metric |
---|---|---|---|
0 | 0 | no scaling | no scaling |
0 | x | no scaling | scale cpu (can burst above 100%) |
x | 0 | scale as per limit | scale as per task limit |
x | x | scale as per limit | scale as per container limit (can burst above 100%) |
Note that unlike docker stats
command and others, CPU usage is not scaled to
the number of CPUs. This means a task with 4 CPUs with all 4 having full
utilization will show up as 400% in docker stats
, but 100% here.
Memory
Memory usage and cache is emitted separately, but the memory usage also includes cache, so subtract cache from it to plot application memory usage specifically.
IO
Network
Network metrics were recently added in Fargate 1.4.0 and ECS agent 1.41.0 onwards.
TODO
[] - Support Non ECS docker host containers
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file ecs-container-exporter-2.3.0.tar.gz
.
File metadata
- Download URL: ecs-container-exporter-2.3.0.tar.gz
- Upload date:
- Size: 12.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38bcb3621febba703bb5d7bb4ca0d150f88a8bcb48feba4bc2c2d6343acdbc54 |
|
MD5 | 28072550a7f4950c1c480d93be0d0b48 |
|
BLAKE2b-256 | 6a7d712993f1425618d3e245bd0c21985ddcc60d5308fb544ff82ecdaa6c5a32 |
File details
Details for the file ecs_container_exporter-2.3.0-py3-none-any.whl
.
File metadata
- Download URL: ecs_container_exporter-2.3.0-py3-none-any.whl
- Upload date:
- Size: 14.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92a599f82cecbd5d6cab90c05686ef445ccc89d71ab89f229f38aae9d9350b10 |
|
MD5 | 4cef4e26a7c2d9664329f639309df243 |
|
BLAKE2b-256 | fe7f063dc3030aa3ea54158cbe0ef7613aae7492a6d222f619663d8b7273c0de |