Monilog
This project allows watching and logging all HTTP traffic of a system, generating log in w3c log format, showing statistics about total requests, maximum hits, requests in a timespan and alerting when traffic is above a customizable threshold.
Requirements
- Python and preferably Linux.
Usage
-
You can execute
simulate.shto run a simulation of how this project works. The simulation setup is customisable, feel free to play with it. -
Make sure to install the package by runing :
python setup.py install
or
pip install monilog
- To run the monitoring in your own log file, run:
monitoring --file /path/to/your/file --threshold 10
- To customize the log generation, run:
log_generator --rates 9 11 8 --durations 150 150 150
With rates being the number of requests per second for each step of the simulation and durations being the durations of the corresponding simulation steps.
- To execute the tests , run :
nosetests --with-coverage --cover-package=monilog
Attention: The monitoring is stopped when no new logs are written to the log
file during MAX_IDLE_TIME set by default to 2 minutes. This is added to manage stopping
the monitoring automatically, particularly when doing limited time simulations.
Future Improvements
This is a first working solution for http log monitoring. Many improvements can be added :
- Managing threaded access to the log file using cross-platform file locking. The current implementation is tested on Linux and may cause errors in Windows.
- Enhancing the display of the log analysis and statistics. For now, the monitoring results are written
to standard output and to a log file with the naming convention
simulation-<timestamp>.log. A better setup would be to customize the GUI using npyscreen for instance. - It is also possible to build a live dashboard consuming
simulation-<timestamp>.logdata and mapping it to graphs. - Pushing alerting notifications by email or SMS to admins / owners of the monitored system.
- Adding more relevant statistics to the analysis of the website and handle timezone changes.
- Writing extensive unit and integration tests.
- For a higher scale of data requiring high availability in a production setup, a more robust solution would be indexing the logs in ElasticSearch and building Kibana dashboards, with a stream-processing platform such as Kafka.
Metadata
Release files for monilog 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| monilog-0.1.4.tar.gz | 6.2 kB | Without provenance | Details for monilog-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| monilog-0.1.4-py3-none-any.whl | Python 3 | none | any | Details for monilog-0.1.4-py3-none-any.whl |
Total release size: 15.4 kB
Release files / monilog-0.1.4.tar.gz
| Download URL | monilog-0.1.4.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2e25f1f1977055c2c9767a2ddc0062cfc11f0e211a8c8a9060bca008a775067f
|
|
BLAKE2b-256 checksum How to use checksums |
27510a36192947eac0eb2e146bdff9d951d23c38391e45020ea6aa4ebb006a35
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3
|
Release files / monilog-0.1.4-py3-none-any.whl
| Download URL | monilog-0.1.4-py3-none-any.whl |
|---|---|
| Size | 9.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3219e6dab625262bb1436cbe14040192a10e9c079c49b39139959eee12fe2ded
|
|
BLAKE2b-256 checksum How to use checksums |
ea7d7194f031785053c300d185f13b827ce1b7e95a68265742d93d7e49d34416
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3
|
Security for monilog 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.