Python bindings for C++ library for reading Prometheus on-disk data
Project description
pypdu
This module provides basic read-only access to the data contained in Prometheus on-disk files from Python.
pypdu
may be installed from pip (on linux and macOS):
pip install pypdu
Example usage:
#!/usr/bin/env python3
import pypdu
data = pypdu.load("/path/to/stats_data")
for series in data:
print(series.name) # equivalent to series.labels["__name__"]
print(series.labels)
print(len(series.samples)) # number of samples can be computed
# without iterating all of them
for sample in series.samples:
print(f"{sample.timestamp} : {sample.value}")
Or the series and samples can be unpacked:
for name, labels, samples in data:
print(name)
print(labels)
print(len(samples))
for timestamp, value in samples:
print(f"{timestamp} : {value}")
Filtering time series
If only a subset of the time series are desired, pypdu
can filter them based on label values, and avoid parsing unneeded series at all:
for series in data.filter({"__name__":"sysproc_page_faults_raw"}):
This will usually perform better than filtering "manually" in python after the fact.
Multiple labels can be specified:
data.filter({"__name__":"sysproc_page_faults_raw", "proc":"memcached"})
ECMAScript regexes can also be used:
data.filter({"proc":pypdu.regex("^go.*")})
Or even arbitrary Python callbacks:
data.filter({"proc":lambda x: x.startswith("go")})
As shorthand, when filtering on __name__
alone, just a string may be provided.
data.filter("sysproc_page_faults_raw")
Single series lookup
If there is only one time series matching your filter, for convenience you can do:
foobar_series = data[{"__name__":"foobar"}]
This is roughly equivalent to:
foobar_series = next(iter(data.filter({"__name__":"foobar"})))
If there are multiple time series matching your filter, this will silently discard all but the lexicographically first (sorted by the key and value of all labels).
If none match, a KeyError
is raised.
All types of filter demonstrated above with .filter(...)
may be used in this manner also.
Histograms
PrometheusData(...).histograms
allows iterating all histograms represented by the time series in a data directory.
The histograms are exposed as HistogramTimeSeries
, grouping all the component ..._bucket
time series together. Indexing into this series provides access to the histogram at a single point in time.
e.g.,
data = pypdu.load("<...>")
for histSeries in data.histograms:
print("Labels: ", histSeries.labels)
print("Number of samples: ", len(histSeries))
for hist in histSeries:
print("TS: ", hist.timestamp)
print(hist.buckets())
Iterates over every histogram found in the Prometheus data, then iterates over every sample contained in that time series.
Example output:
Labels: {'__name__': 'cm_http_requests_seconds', 'instance': 'ns_server', 'job': 'ns_server_high_cardinality'}
Number of samples: 3826
TS: 1621268098827
[(0.001, 8.0), (0.01, 25.0), (0.1, 25.0), (1.0, 25.0), (10.0, 25.0), (inf, 25.0)]
TS: 1621268158827
[(0.001, 39.0), (0.01, 118.0), (0.1, 126.0), (1.0, 127.0), (10.0, 127.0), (inf, 127.0)]
TS: 1621268218827
[(0.001, 43.0), (0.01, 132.0), (0.1, 140.0), (1.0, 141.0), (10.0, 141.0), (inf, 141.0)]
TS: 1621268278827
[(0.001, 48.0), (0.01, 145.0), (0.1, 153.0), (1.0, 154.0), (10.0, 154.0), (inf, 154.0)]
TS: 1621268338827
[(0.001, 53.0), (0.01, 158.0), (0.1, 166.0), (1.0, 167.0), (10.0, 167.0), (inf, 167.0)]
TS: 1621268398827
[(0.001, 55.0), (0.01, 171.0), (0.1, 179.0), (1.0, 180.0), (10.0, 180.0), (inf, 180.0)]
TS: 1621268458827
[(0.001, 60.0), (0.01, 191.0), (0.1, 199.0), (1.0, 200.0), (10.0, 200.0), (inf, 200.0)]
TS: 1621268518827
[(0.001, 66.0), (0.01, 204.0), (0.1, 212.0), (1.0, 213.0), (10.0, 213.0), (inf, 213.0)]
TS: 1621268578827
[(0.001, 71.0), (0.01, 217.0), (0.1, 225.0), (1.0, 226.0), (10.0, 226.0), (inf, 226.0)]
TS: 1621268638827
[(0.001, 73.0), (0.01, 230.0), (0.1, 238.0), (1.0, 239.0), (10.0, 239.0), (inf, 239.0)]
...
Labels: ...
HistogramTimeSeries
(in the above example, this is histSeries
), can be indexed into - currently
only by a sample index, but in the future, selecting the histogram closest to a given timestamp may be supported.
E.g., the first and last point in time view available for a specific histogram can be found with:
first = histSeries[0]
last = histSeries[-1]
From which the timestamp and buckets could be read:
>>> print(last.timestamp) # time since epoch in ms
1631007596974
>>> print(last.bucket_bounds()))
[0.001, 0.01, 0.1, 1.0, 10.0, inf]
>>> print(last.bucket_values())
[4279.0, 4371.0, 4666.0, 5044.0, 5044.0, 5044.0]
>>> print(last.buckets()) # convenience zip of (bounds, values)
[(0.001, 4279.0), (0.01, 4371.0), (0.1, 4666.0), (1.0, 5044.0), (10.0, 5044.0), (inf, 5044.0)]
Alternative installation steps
setup.py
pypdu
may be installed without pip
. To use, clone the repository as in the installation instructions.
Then run:
python setup.py install
This should also work on Windows, but is untested.
manual .so
Alternatively, following the cmake
steps in the installation instructions to build the project produces a module with a platform-dependent name - for example on MacOS this may be pypdu.cpython-39-darwin.so
.
This can be found either in <build dir>/src/pypdu
or in your chosen installation prefix. This can be used without installing with setup.py
, simply ensure the containing directory is in your PYTHONPATH
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for pypdu-0.0.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ee9c6fa58ef2125f94dc5ce1c3f307742a40b516f99844c318a59094b1fd649 |
|
MD5 | 3186506e49ff806fe39103cf18cfc037 |
|
BLAKE2b-256 | 434ab7105827c4320a72a0dc3817449b4ca9bab0e332762022f6c3c26e3a3bf6 |
Hashes for pypdu-0.0.10-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | deeef0f8a1c5b7e64f1cc54dfe9506a6d48a4f9d107829fc200cee29810cf6fd |
|
MD5 | 8d9b5ce0db6e8182f75945297780e1f1 |
|
BLAKE2b-256 | d51bcbbe53163213ae14c9b24c1670519fe5cdd4861f661f8f9ed02a65218f2f |
Hashes for pypdu-0.0.10-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b839ede6ede1136b05c215c17858c5919d33ecc45964c6f2f4d74a4590e82c5 |
|
MD5 | 0d4fa9e3758729b63dc8402af3720a8e |
|
BLAKE2b-256 | acb938519a50ea1739b42bcd111e9468b0b909a703c4c6cb295c68b09821d453 |
Hashes for pypdu-0.0.10-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e211789505b54d303a44afbd4e24e17b0f7546c81b9e4bc4d3a69946f721fc31 |
|
MD5 | d22cde7135e34f42fffacfa5cb94c607 |
|
BLAKE2b-256 | 98cc5340299f15731d1794721b23a080419512b64d04785e9795cebe620185f8 |
Hashes for pypdu-0.0.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb02e1b0781d8b271cee16c6cfab8a24e1c65ea924bab080b54ef00562d04a8f |
|
MD5 | c9115502c012dc6cb726690d2e0bf792 |
|
BLAKE2b-256 | 26c9a376487650117d261566499a5d835eedbfbf784e82a40a9a6035f5ebb6e7 |
Hashes for pypdu-0.0.10-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1b88d13feff832805e1431b06a9c4221f1b8079eac748639cb694bed132b64d4 |
|
MD5 | cb88e36c260d463af9d2eee90383419e |
|
BLAKE2b-256 | e6aac6fc34ca49c3c6cb4363e7e0d1c2c3e1cbecb504cac0da29c8e0f0ba6169 |
Hashes for pypdu-0.0.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e027617fdfa652d108899da50c42bc786d9298095bcfc99e94f895a4d9b8cfe6 |
|
MD5 | 73e77bdc64d48349d4c83bbc44d913c5 |
|
BLAKE2b-256 | 37fbad3cd6bb108b3f2dcb0670b135362843efb039b994140be2cd42504c96ea |
Hashes for pypdu-0.0.10-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e0f68f584720b1f5ee7bac8e073285d0b4cc11212f4da8519cd5d0bef0056af |
|
MD5 | f6c295c386fb952fe4e145ed6448eb1d |
|
BLAKE2b-256 | 9274199f876d8d7bcb0b2c9f8f53c00453143f644f9f36d4399a2cd1f3a1a6f4 |
Hashes for pypdu-0.0.10-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 433d91b06effbef94c26327f3790250cad154bbb284ea76310d6ba5952fa5e60 |
|
MD5 | 767bd32eade4cdc0fd3246433881affa |
|
BLAKE2b-256 | 1e46b608998e313ec310c44247a52af83036f04e7b45c68b692dbee3d35a7ca3 |
Hashes for pypdu-0.0.10-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e41f9e594c01e438f0827bb80dd6d4098b1c3d8529cbce65d582345c8712ab9 |
|
MD5 | b505c33ccbc96e90d78b5f157fcbd542 |
|
BLAKE2b-256 | 676599fce189eafa435195f67be6b02446c797e91807afbb97d7a1d10f5e6745 |
Hashes for pypdu-0.0.10-cp27-cp27mu-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4b55faea8cf3c4e72a9c4721aeddf2eb5593e7ea5cb646822584bde001b7e63 |
|
MD5 | e9133ed2ffd37fdd2b62df86eebb0024 |
|
BLAKE2b-256 | 54492e0ef8da8b7b74421bee3622d1c387759876d7a04d2917beac915b36ffd2 |
Hashes for pypdu-0.0.10-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 34d7d348b181ef1482cddc229295ddc0dc0bd0ab8a03d870eb31930feb5d3356 |
|
MD5 | a272ff41e9ba3b4af4b7a1b95a6f2fdc |
|
BLAKE2b-256 | 5e94f6f040ddf9491029f373762b0304e645e1179d7ec7c1777675ca43850666 |