Skip to main content

Export logs from ringserver as postgresql timeseries

Project description

ringserverstats

Parse ringserver reports, compute statistics and store in influxdb.

ringserver activity

ringserver statistics by day

ringserver statistics by month

Installation

ringserverstats is distributed on PyPI https://pypi.org as a universal wheel.

    $ pip install ringserverstats

Postgresql preparation

CREATE ROLE resifstats WITH LOGIN PASSWORD 'md5xxxxxxxxxx';
CRATE DATABASE resifstats OWNER resifstats;
\c resifstats
CREATE EXTENSION timescaledb

Then go for the migrations :

$ yoyo apply --database postgresql://resifstats@resif-pgdev.u-ga.fr:5433/resifstats -m migrations

Docker

docker build -t ringserverstats:latest
docker run --rm -p 8000:8000 -e DATABASE_URI=postgresql://resifstats@resif-pgdev.u-ga.fr:5433/resifstats ringserverstats:latest

Usage

To work properly, this program needs the environment variable DATABASE_URI pointing to a valip postgresql URI

Command line interface

$ pip install --user ringserverstats
$ ringserverstats txlogs.log

Web application

For development :

cd webapp
FLASK_ENV=development FLASK_APP=webapp.py flask run

In production, use with gunicorn :

pip install gunicorn
cd ringserverstats
gunicorn -b 0.0.0.0:8000 webapp

To send date :

http localhost:8000 < txlogs.log
wget --post-file txlogs.log localhost:8000

Explanations

The TX logs from ringserver are metrics suitable for a timeserie database. The idea is to parse the logs, as in the exemple below, and to generate values to insert into an influxdb timeseries database.

The file grafana-dashboard.json can be imported into grafana to visualize this timeserie.

Events

The ringserverevents measure has several tags :

  • network, station, location, channel : which data was requested
  • geohash : location of the client in geohash format
  • hosthash : a hash of the client ip (usefull to correlate the clients requests)
  • city : an english city name

Grouping and downsampling

To achieve downsampling, we use timescaledb's continuous query.

License

ringserverstats is distributed under the terms of the GPL v3 or later. See LICENSE file.

Build

python3 setup.py sdist bdist_wheel

Test

tox

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ringserverstats, version 0.14.4
Filename, size File type Python version Upload date Hashes
Filename, size ringserverstats-0.14.4.tar.gz (6.0 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page