vuakhter 0.0.5

Package to count usage statistics from ELK logs.

Vuakhter

Vuakhter is validation tool to test API to conform our API guide. But it can be used to generate statistics of web-application by its logs.

Principles of work

Vuakhter scans access log for specified period, filters log entries by path prefixes and passes each entry for statistics generation.

Base class AccessLog returns an iterator of log entries. Class ElasticAccessLog scans elastic indices for log entries.

StatisticsMetrics gets AccessEntry records and forms array of statistics data. StatisticsMetrics.report() -> str returns statistics report for the metrics.

HttpAnalyzer uses access_log: AccessLog and array of StatisticsMetrics.

Only one metric SchemaValidatorCounter is passing to HttpAnalyzer in main script vuakhter. It uses request_log: RequestLog to validate API responses.

Class ElasticRequestLog: RequestLog scans elastic indices requests by request_id and returns array of RequestEntry. SchemaValidatorCounter checks all responses bodies and counts valid API calls.

Installation

pip install vuakhter

Usage

vuakhter [--es-user ES_USER] [--es-pass ES_PASS] [--es-host ES_HOST] [--es-port ES_PORT
[--start-date START_DATE] [--end-date END_DATE] prefixes [prefixes ...]

By default end_date is current date and time if not specified. And start_date defaults to day ago end_date.

All connection parameters may be specified in .env file.

ES_USER=elastic ES_PASS=pasword ES_HOST=localhost vuakhter /api/

Using in code

import datetime

from elasticsearch import Elasticsearch

from vuakhter.analyzer import HttpAnalyzer
from vuakhter.kibana.access_log import ElasticAccessLog
from vuakhter.metrics.counters import ComplexCounter


elastic = Elasticsearch()
access_log = ElasticAccessLog(index_pattern='filebeat-*', client=elastic)

http_analyzer = HttpAnalyzer(access_log=access_log)
http_analyzer.add_metrics(ComplexCounter())

end_date = datetime.datetime.now()
start_date = end_date - datetime.timedelta(days=1)

http_analyzer.analyze(start_date, end_date)

for metric in http_analyzer.metrics:
    metric.finalize()
    print(metric.report())

Contributing

We would love you to contribute to our project. It's simple:

• Create an issue with bug you found or proposal you have. Wait for approve from maintainer.
• Create a pull request. Make sure all checks are green.
• Fix review comments if any.
• Be awesome.

Here are useful tips:

• You can run all checks and tests with make check. Please do it before TravisCI does.
• We use BestDoctor python styleguide.
• We respect Django CoC. Make soft, not bullshit.