Skip to main content

Monilog

Build Status codecov PyPI Version

This project allows watching and logging all HTTP traffic of a system, generating log in w3c log format, showing statistics about total requests, maximum hits, requests in a timespan and alerting when traffic is above a customizable threshold.

Requirements

  • Python and preferably Linux.

Usage

  • You can execute simulate.sh to run a simulation of how this project works. The simulation setup is customisable, feel free to play with it.

  • Make sure to install the package by runing :

python setup.py install 

or

pip install monilog
  • To run the monitoring in your own log file, run:
monitoring --file /path/to/your/file --threshold 10
  • To customize the log generation, run:
log_generator --rates 9 11 8 --durations 150 150 150

With rates being the number of requests per second for each step of the simulation and durations being the durations of the corresponding simulation steps.

  • To execute the tests , run :
nosetests --with-coverage --cover-package=monilog

Attention: The monitoring is stopped when no new logs are written to the log file during MAX_IDLE_TIME set by default to 2 minutes. This is added to manage stopping the monitoring automatically, particularly when doing limited time simulations.

Future Improvements

This is a first working solution for http log monitoring. Many improvements can be added :

  • Managing threaded access to the log file using cross-platform file locking. The current implementation is tested on Linux and may cause errors in Windows.
  • Enhancing the display of the log analysis and statistics. For now, the monitoring results are written to standard output and to a log file with the naming convention simulation-<timestamp>.log. A better setup would be to customize the GUI using npyscreen for instance.
  • It is also possible to build a live dashboard consuming simulation-<timestamp>.log data and mapping it to graphs.
  • Pushing alerting notifications by email or SMS to admins / owners of the monitored system.
  • Adding more relevant statistics to the analysis of the website and handle timezone changes.
  • Writing extensive unit and integration tests.
  • For a higher scale of data requiring high availability in a production setup, a more robust solution would be indexing the logs in ElasticSearch and building Kibana dashboards, with a stream-processing platform such as Kafka.

Metadata

Release files for monilog 0.1.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for monilog 0.1.4
File Size Uploaded Provenance Actions
monilog-0.1.4.tar.gz 6.2 kB Without provenance Details for monilog-0.1.4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for monilog 0.1.4
File Interpreter ABI Platform Actions
monilog-0.1.4-py3-none-any.whl Python 3 none any Details for monilog-0.1.4-py3-none-any.whl

Total release size: 15.4 kB

Release files / monilog-0.1.4.tar.gz

Download URL monilog-0.1.4.tar.gz
Size 6.2 kB
Tags Source
SHA-256 checksum
How to use checksums
2e25f1f1977055c2c9767a2ddc0062cfc11f0e211a8c8a9060bca008a775067f
BLAKE2b-256 checksum
How to use checksums
27510a36192947eac0eb2e146bdff9d951d23c38391e45020ea6aa4ebb006a35
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3

Release files / monilog-0.1.4-py3-none-any.whl

Download URL monilog-0.1.4-py3-none-any.whl
Size 9.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
3219e6dab625262bb1436cbe14040192a10e9c079c49b39139959eee12fe2ded
BLAKE2b-256 checksum
How to use checksums
ea7d7194f031785053c300d185f13b827ce1b7e95a68265742d93d7e49d34416
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.7.3

Security for monilog 0.1.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page