nx-logstats 0.1.2

A tool for analyzing NGINX access logs

nx-logstats: NGINX Log Analysis Tool

A NGINX log analysis tool that parses a given log file (e.g., NGINX access logs), summarizes key metrics such as request volume, status code distribution, frequently accessed endpoints, outputs a clean report and number of requests per hour of the day to showcase the temporal traffic patterns.

Overview

nx-logstats is a command-line utility that parses NGINX access logs, calculates key metrics, and presents them in a readable format. It's designed to be straightforward to use while providing meaningful insights from your server logs. Output can be in JSON or simple text. It can be shown in the terminal or stored in a file such as results.json.

Key features:

• Parse standard NGINX access log formats
• Extract multiple metrics from log data:
  • HTTP status code distribution
  • Most frequently accessed endpoints
  • Request volume by hour
  • HTTP method distribution
  • Average response size
• Output to terminal or file in text or JSON formats
• Robust error handling for malformed log entries
• Different Logging levels that can be configured with options -v for info level logs and -vv for debug level logs.
• Configured with Github Workflow which publishes the artifact to PyPi repository so that it can be accessed by anyone globally, available at https://pypi.org/project/nx-logstats/ and https://github.com/manvirheer/nx-logstats/actions.

Installation

From PyPI (Not suitable for running tests but packaged for production usage)

pip install nx-logstats

From Source (Preferred for running tests)

git clone https://github.com/manvirheer/nx-logstats.git
cd nx-logstats

Suggestion - Create a Virtual Environment

For Windows

python -m venv venv
venv\Scripts\activate

For Linux/MacOS

python -m venv venv
source venv/bin/activate

For development and testing:

pip install -e ".[dev]"

Usage

Basic usage:

There is a sample_logs/ folder available in the repository which can be used to test the solution.

nx-logstats ./sample_logs/valid_simple_logs.txt
nx-logstats ./sample_logs/errors_small_dataset.txt
nx-logstats ./sample_logs/errors_small_dataset.txt --ignore-errors
nx-logstats ./sample_logs/errors_small_dataset.txt --ignore-errors -f json -o results.json
nx-logstats ./sample_logs/errors_small_dataset.txt -v
nx-logstats ./sample_logs/errors_small_dataset.txt -vv

Command Line Options

usage: nx-logstats [-h] [-o OUTPUT] [-f {text,json}] [-n TOP_N] [-v] [--ignore-errors] logfile

NGINX log file analyzer - extracts and reports on key metrics. Accepted format is: <ip> - - [<dd/Mon/YYYY:HH:MM:SS>] "<HTTP_METHOD> <path> HTTP/1.1" <status> <bytes>

positional arguments:
  logfile               Path to the NGINX access log file to analyze

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Path to write the output report (default: print to stdout)
  -f {text,json}, --format {text,json}
                        Output format (text or json) (default: text)
  -n TOP_N, --top-n TOP_N
                        Number of top endpoints to include in the report (default: 10)
  -v, --verbose         Increase verbosity: -v for INFO, -vv for DEBUG. Default shows only errors. (default: 0)
  --ignore-errors       Ignore malformed log lines and continue processing (default: False)

Sample Output

When using the text format, the output will look similar to:

┌─────────────────────── NGINX ACCESS LOG ANALYSIS REPORT ───────────────────────┐
└──────────────────── Generated at: 2025-April-06 12:34:56 ────────────────────┘
┌─────────────────────────── General Statistics ───────────────────────────┐
│ Total Requests:       1242                                               │
│ Average Response Size: 4231.76 bytes                                     │
│ Generated at:         2025-April-06 12:34:56                             │
└────────────────────────────────────────────────────────────────────────┘
┌─── HTTP Status Code Distribution ────┐  ┌───── HTTP Method Distribution ─────┐
│ Status │ Count    │ Percentage       │  │ Method │ Count    │ Percentage     │
│ 200    │ 1024     │ 82.4%            │  │ GET    │ 985      │ 79.3%          │
│ 404    │ 156      │ 12.6%            │  │ POST   │ 192      │ 15.5%          │
│ 500    │ 42       │ 3.4%             │  │ PUT    │ 38       │ 3.1%           │
│ 302    │ 20       │ 1.6%             │  │ DELETE │ 27       │ 2.2%           │
└──────────────────────────────────────┘  └────────────────────────────────────┘

...
...
...

Running Tests

Run the test suite with:

pytest

License

This project is licensed under the MIT License - see the LICENSE file for details.

Design Considerations

For the design considerations, review the Design.md