Prometheus HTTP SD framework.
Project description
prometheus-http-sd
This is a Prometheus HTTP SD framework.
Features
- Support static targets from Json file;
- Support static targets from Yaml file;
- Support generating target list using Python script;
- Support
check
command, to testing the generated target is as expected, and counting the targets.
Installation
pip install prometheus-http-sd
Usage
First, you need a directory, everything in this directory will be used to generate targets for prometheus-http-sd.
$ mkdir targets
In this directory:
- Filename that ending with
.json
will be exposed directly - Filename that ending with
.yaml
will be exposed directly - Filename that ending with
.py
must include agenerate_targets()
function, the function will be run, and it must return aTargetList
(Type helper inprometheus_http_sd.targets.
) - Filename that starts with
_
will be ignored, so you can have some python utils there, for e.g._utils/__init__.py
that you can import in yougenerate_targets()
- Filename that starts with
.
(hidden file in Linux) will also be ignored
Then you can run prometheus-http-sd -h 0.0.0.0 -p 8080 /tmp/targets
,
prometheus-http-sd will start to expose targets at: http://0.0.0.0:8080/targets
The -h
and -p
is optional, defaults to 127.0.0.1
and 8080
.
$ prometheus-http-sd /tmp/good_root
[2022-07-24 00:52:03,896] {wasyncore.py:486} INFO - Serving on http://127.0.0.1:8080
Check and Validate your Targets
You can use prometheus-http-sd check
command to test your targets dir. It will
run all of you generators, validate the targets, and print the targets count
that each generator generates.
$ prometheus-http-sd check test/test_generator/root
[2022-08-06 00:50:11,095] {validate.py:16} INFO - Run generator test/test_generator/root/json/target.json, took 0.0011398792266845703s, generated 1 targets.
[2022-08-06 00:50:11,100] {validate.py:16} INFO - Run generator test/test_generator/root/yaml/target.yaml, took 0.0043718814849853516s, generated 2 targets.
[2022-08-06 00:50:11,100] {validate.py:22} INFO - Done! Generated {total_targets} in total.
It's a good idea to use prometheus-http-sd check
in your CI system to validate
your targets generator scripts and target files.
Script Dependencies
If you want your scripts to use some other python library, just install them into the same virtualenv that you install prometheus-http-sd, so that prometheus-http-sd can import them.
The Target Path
prometheus-http-sd support sub-pathes.
For example, if we use export PROMETHEUS_HTTP_SD_DIR=gateway
, and the
gateway
directory's structure is as follows:
gateway
├── nginx
│ ├── edge.py
│ └── targets.json
└── targets.json
Then:
/targets/gateway
will return the targets from:gateway/nginx/edge.py
gateway/nginx/targets.json
gateway/targets.json
/targets/gateway/nginx
will return the targets from:gateway/nginx/edge.py
gateway/nginx/targets.json
This is very useful when you use vertical scaling. Say you have 5 Prometheus instances, and you want each one of them scrape for different targets, then you can use the sub-path feature of prometheus-http-sd.
For example, in one Prometheus's config:
scrape_configs:
- job_name: "nginx"
http_sd_config:
url: http://prometheus-http-sd:8080/targets/nginx
- job_name: "etcd"
http_sd_config:
url: http://prometheus-http-sd:8080/targets/etcd
And in another one:
scrape_configs:
- job_name: "nginx"
http_sd_config:
url: http://prometheus-http-sd:8080/targets/database
- job_name: "etcd"
http_sd_config:
url: http://prometheus-http-sd:8080/targets/application
Update Your Scripts
If you want to update your script file or target json file, just upload and overwrite with your new version, it will take effect immediately after you making changes, there is no need to restart prometheus-http-sd, prometheus-http-sd will read the file (or reload the python script) every time serving a request.
It is worth noting that restarting is safe because if Prometheus failed to get the target list via HTTP request, it won't update its current target list to empty, instead, it will keep using the current list.
Prometheus caches target lists. If an error occurs while fetching an updated targets list, Prometheus keeps using the current targets list.
For the same reason, if there are 3 scripts under /targets/mysystem
and only
one failed for a request, prometheus-http-sd will return a HTTP 500 Error for
the whole request instead of returning the partial targets from the other two
scripts.
Also for the same reason, if your script met any error, you should throw out
Exception
all the way to the top instead of catch it in your script and return
a null TargetList
, if you return a null TargetList
, prometheus-http-sd will
think that your script run successfully and empty the target list as well.
You can notice this error from stdout logs or /metrics
from
prometheus-http-sd.
Best Practice
You can use a git repository to manage your target generator.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for prometheus_http_sd-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f73c19f271d66e57ebe726d52b83c30f83f2ca26d293827935357a77d280688 |
|
MD5 | a2c3e684ff4257064533ed4e0479a421 |
|
BLAKE2b-256 | 553d27359f6d7f86132b8383c7bfd662646f1df5defa9db9907fc512ea07018a |