Export logs from ringserver as influxdb timeseries
Parse ringserver reports, compute statistics and store in influxdb.
ringserverstats is distributed on PyPI https://pypi.org as a universal wheel.
$ pip install ringserverstats
This program will fill 2 measurements. You should configure a database for these, and configure a user with write priviledges.
First, create a database, a user, and retention policie.
create database ringserverdb create user ringserver with password 'ringserverer' grant write on ringserverdb to ringserver grant read on ringserverdb to ringserver create retention policy rp_ringserverevents on ringserver duration 1w replication 1 create retention policy rp_ringserverstats on ringserver duration 520w replication 1
To work properly, this program needs the following environment variables set :
INFLUXDB_HOST: The host name or adress of influxdb server
INFLUXDB_PORT: The port number of influxdb server
INFLUXDB_USER: The influxdb user to authenticate to
INFLUXDB_PASS: The password to authenticate with
INFLUXDB_DB: The database name containing the metric
INFLUXDB_VERIFY_SSL: Set to
noto verify SSL connection
INFLUXDB_SSL: Should the connection go to https ?
$ ringserstats txlogs.log
The TX logs from ringserver are metrics suitable for a timeserie database. The idea is to parse the logs, as in the exemple below, and to generate values to insert into an influxdb timeseries database.
grafana-dashboard.json can be imported into grafana to visualize this timeserie.
Used tags in influxdb :
show tag keys from ringserverevents show tag keys from ringserverstats
The ringserverevents measure has several tags :
- network, station, location, channel : which data was requested
- geohash : location of the client in geohash format
- hosthash : a hash of the client ip (usefull to correlate the clients requests)
- city : an english city name
Grouping and downsampling
Because storing events in the longterm is not very relevant, the
ringserverstats measurement groups all events by day, by host, by network, station, location and channel.
The retention policies discribed above will manage the time your data get stored in influxdb.
To achieve downsampling, we use influxdb's continuous query. This is an exemple of creating the continuous query :
create continuous query "cq_ringserverstats_by_day" on resif resample for 3d begin select sum("bytes") as bytes into "rp_ringserverstats"."ringserverstats" from "rp_ringserverevents"."ringserverevents" group by time(1d),network,station,location,channel,city,hosthash,geohash end
This query is executed every day (from the
group by time(1d) instruction) and will resample 3 days in the past (instruction
resample for 3 d), genrating 3 entries (one per day). Existing entries are overwritten.
Documentation of influxdb's continuous queries is here : https://docs.influxdata.com/influxdb/v1.7/query_language/continuous_queries/
ringserverstats is distributed under the terms of the GPL v3 or later. See LICENSE file.
python3 setup.py sdist bdist_wheel
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size ringserverstats-0.9.0.tar.gz (5.8 kB)||File type Source||Python version None||Upload date||Hashes View hashes|