Export logs from rsync daemon logs as influxdb timeseries
Parse rsync server logs, compute statistics and store in influxdb.
ringserverstats is distributed on PyPI https://pypi.org as a universal wheel.
$ pip install rsyncstats
This program will fill 2 measurements. You should configure a database for these, and configure a user with write priviledges.
First, create a database, a user, and retention policie.
create database rsyncdb create user rsync with password 'rsyncer' grant write on rsyncdb to rsync grant read on rsyncdb to rsync create retention policy rp_rsyncevents on rsync duration 1w replication 1 create retention policy rp_rsyncstats on rsync duration 520w replication 1
To work properly, this program needs the following environment variables set :
INFLUXDB_HOST: The host name or adress of influxdb server
INFLUXDB_PORT: The port number of influxdb server
INFLUXDB_USER: The influxdb user to authenticate to
INFLUXDB_PASS: The password to authenticate with
INFLUXDB_DB: The database name containing the metric
INFLUXDB_VERIFY_SSL: Set to
noto verify SSL connection
INFLUXDB_SSL: Should the connection go to https ?
$ ringserstats txlogs.log
The logs from rsync are metrics suitable for a timeserie database. The idea is to parse the logs, as in the exemple below, and to generate values to insert into an influxdb timeseries database.
grafana-dashboard.json can be imported into grafana to visualize this timeserie.
Used tags in influxdb :
show tag keys from rsyncevents show tag keys from rsyncstats
The rsyncevents measure has several tags :
- module : the rsync moduled accesed during transfer
- geohash : location of the client in geohash format
- hosthash : a hash of the client ip (usefull to correlate the clients requests)
- city : an english city name
Because storing events in the longterm is not very relevant, the
rsyncstats measurement groups all events by day, by host and by network. The retention policies discribed above will manage the time your data get stored in influxdb.
rsyncstats is distributed under the terms of the GPL v3 or later. See LICENSE file.
python3 setup.py sdist bdist_wheel
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size rsyncstats-0.1.5-py3-none-any.whl (17.7 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size rsyncstats-0.1.5.tar.gz (5.6 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for rsyncstats-0.1.5-py3-none-any.whl