LogLens

Log analysis and anomaly detection for application logs. LogLens parses plain text and JSON logs, groups errors by signature, finds repeated failures and bursts, flags time windows that deviate from the file's own baseline, reduces everything to a single health score, and keeps a history so runs can be compared over time.

It has no dependencies. Terminal output is in Turkish.

Install

Python 3.10 or newer. No third-party dependencies.

git clone https://github.com/LocalinTheEngineer/loglens
cd loglens
python main.py sample_logs/server.log

Or install it from PyPI:

pip install loglynx
loglynx app.log          # loglens works too

The distribution is published as loglynx because loglens is too close to an existing PyPI name. Both commands are installed and do the same thing; the module and the repository stay loglens.

From a checkout, to put the loglens command on your PATH:

pip install .
loglens sample_logs/server.log

Or run it in a container, with the dashboard as the entry point:

docker compose up -d                 # http://localhost:8000
docker run --rm -v "$PWD/logs:/logs" loglens:1.1.1 /logs/app.log

The examples below use python main.py; loglens works identically.

Usage

python main.py app.log                          analyse one file
python main.py logs/                            a directory, .gz included
python main.py "logs/*.log.gz"                  a glob
cat app.log | python main.py -                  stdin

python main.py app.log --since 18:00 --until 20:00
python main.py app.log --grep timeout --level ERROR --logger db

python main.py app.log --html report.html       standalone HTML report
python main.py app.log --json                   machine-readable output
python main.py app.log --quiet                  score and findings only

python main.py app.log --save --label "after deploy"
python main.py --history                        past runs
python main.py --compare                        last two runs
python main.py --compare 3:7                    two runs by id

python main.py app.log --watch                  follow the file live
python main.py --serve                          dashboard on localhost:8000

Rotated files are merged and sorted chronologically, so app.log, app.log.1 and app.log.2.gz can be analysed as one timeline. Encoding is detected automatically. Multi-line tracebacks are attached to the entry they belong to instead of being discarded.

Sample output:

╔══════════════════════════════════════════════════════╗
║ SYSTEM HEALTH         44.8 / 100   [D]               ║
║ █████████████░░░░░░░░░░░░░░░░░  Sorunlu              ║
╠══════════════════════════════════════════════════════╣
║ -35.0  Hata oranı            ▪▪▪▪▪▪▪▪▪▪              ║
║ -10.0  Anomaliler            ▪▪▪▪▪·····              ║
║  -5.2  Kritik kayıtlar       ▪▪▪·······              ║
╚══════════════════════════════════════════════════════╝

── SAATLİK HATA DAĞILIMI ───────────────────────────────
17 ──                                        4
18 ── ██████████████████████████████████    229 ← zirve
19 ── ███████████████                       105

┌─ Error cluster #1 ────────────────────────────────────
│ DatabaseConnectionError
│ Occurrences : 275  (4.2/dk)
│ First seen  : 18:15:00
│ Last seen   : 19:19:57  (1s 4dk sürdü)
│ Module      : db
└──────────────────────────────────────────────────────

What it looks like in practice

A database outage. You are handed yesterday's log and told "something happened around six".

python main.py app.log --quiet

The score comes back 44.8/100 (D), and the findings name the incident without you reading a single line: 229 errors in the 18:00 hour against a baseline of 4, one signature accounting for 72% of them, the db module at a 59% error rate, and — the part that is easy to miss by eye — a twelve minute stretch at 22:47 with no log output at all. Running it again without --quiet prints the cluster: DatabaseConnectionError, 275 occurrences, 18:15:00 to 19:19:57, with a rule-based guess at the cause.

A deploy that made things worse. Save a run before the deploy and another after, and the second one prints the difference:

python main.py app.log --save --label "before"
# deploy
python main.py app.log --save --label "after"

Sağlık skoru     87.9 → 44.8  ▼ (-43.1)   KÖTÜLEŞME
Yeni ortaya çıkan hatalar:
  + CircuitBreakerOpenError (20)

CircuitBreakerOpenError did not exist in the earlier run. That is not a guess: error signatures are stored per run, so "new" means literally absent from the previous set. As a CI gate:

python main.py app.log --fail-on health --min-health 80

A spike of 5xx on the front end. Point it at the access log and narrow to the window you care about:

python main.py /var/log/nginx/access.log --since 18:00 --until 20:00 --level ERROR

Access logs have no level field, so the HTTP status supplies one — 5xx becomes ERROR, 4xx WARNING. The top-errors table then groups by request shape rather than by URL, because ids and query strings are masked out of the signature.

Formats

The format is detected per file, and lines that do not match the dominant format fall back to the other parsers, so mixed files still work.

JSON field names are not standardised, so the known spellings are all tried: OpenTelemetry (severity_text, body), Elastic ECS (log.level, @timestamp, service.name) and Python's own (levelname, asctime, name). Unrecognised keys are kept on the entry. Timestamps may be ISO strings or epoch numbers; offset-aware values are normalised to UTC so a file mixing both does not break comparisons.

Two of these carry no level field, so one is derived. Syslog encodes severity in the priority number (<11> is facility 1, severity 3, an error), and it is decoded from there; a BSD line without a priority falls back to a level word at the start of the message, and INFO otherwise. Access logs have a status code instead: 5xx becomes ERROR, 4xx WARNING, the rest INFO. RFC 3164 also omits the year, so the current one is assumed.

Adding a format means adding one class with an extract(line) method.

Anomaly detection

Anomalies are scored against the file's own baseline rather than a fixed threshold, because "too many errors" only means something relative to what that system normally produces.

The default is a modified z-score, 0.6745 * (x - median) / MAD. The obvious alternative, (x - mean) / stdev, has a flaw that matters here: the outlier you are looking for inflates the standard deviation and therefore hides itself. In the sample log there are two error spikes, at 18:00 and 19:00. The classic score only reports the first; the larger spike raises stdev to 49.6 and the second one falls inside the noise. The median and MAD are unaffected by outliers, so both are reported. tests/test_anomalies.py::test_robust_beats_classic_zscore pins this down. The classic method is still available with --method zscore.

Long silences are treated as anomalies too. A stopped service or a dead log agent produces nothing at all, which error counting never catches.

Health score

A single 0-100 number, computed by subtracting penalties from 100. Each factor has a ceiling, so one bad metric cannot flatten the score and make every bad log look identical.

Grades: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F below. The weights are a judgement call, not a derived truth; WEIGHTS in src/analyzer/health.py is the one place to retune them.

Live mode

--watch follows a file the way tail -f does and redraws a panel as lines arrive, raising an alert on CRITICAL entries, on error bursts, and the first time an error signature appears.

"First time seen" only means something once there is a baseline, so signature alerts stay off during a warm-up period (50 entries and 30 seconds of log time). Without it every signature is new at startup and the panel fills with alerts that carry no information. Bursts and CRITICAL entries alert immediately, since neither depends on history.

╔══════════════════════════════════════════════════════╗
║ LOGLENS LIVE  ·  15 dakikalık pencere                ║
╠══════════════════════════════════════════════════════╣
║ Pencere 18:15:10 → 18:15:30                          ║
║ Toplam  13 kayıt  (başlangıçtan beri 13)             ║
║ Hata    %100.0   son 60sn: 13                        ║
║ Sağlık  42 / 100  [D]                                ║
╠══════════════════════════════════════════════════════╣
║ DatabaseConnectionError                 12           ║
╚══════════════════════════════════════════════════════╝

Uyarılar:
  18:15:19 [BURST] 10 hata / 60 saniye
  18:15:30 [CRITICAL] Service unavailable: all database replicas down

Analysis runs over a rolling window (--window, 15 minutes by default), so the numbers describe what is happening now rather than the whole file. File rotation and truncation are detected by watching the inode and the file size, so the follower survives logrotate and > app.log.

To try it without a live service, replay an existing file into another one from a second terminal:

python tools/replay_log.py sample_logs/server.log sample_logs/live_test.log --speed 5
python main.py sample_logs/live_test.log --watch

History

--save writes the run to SQLite (loglens.db by default, --db to change it) and prints the difference against the previous run:

── ÖNCEKİ ANALİZLE KARŞILAŞTIRMA ───────────────────────
#1 2026-08-23 16:17  →  #2 2026-08-23 16:17

  Sağlık skoru     87.9 → 44.8  ▼ (-43.1)   KÖTÜLEŞME
  Hata oranı        8.7% → 30.0%  ▲ (+21.4 puan)
  Anomali             0  → 2  ▲ (+2)

  Yeni ortaya çıkan hatalar:
    + CircuitBreakerOpenError (20)
    + Service unavailable: all database replicas down (3)

  Sayısı değişenler:
    ▲ DatabaseConnectionError: 1 → 293 (+29200%)

Error signatures are stored per run, which is what makes "this error type is new" and "this one is gone" answerable rather than guesswork. --prune N keeps the newest N runs.

Note that SQLite needs real file locking, so a database on a network share or a syncing folder may fail to open; pass --db with a local path in that case.

AI commentary

Everything above is computed in Python. --explain adds an optional layer on top: the findings are handed to a language model, which writes the incident up in prose — probable root cause, alternative explanations, what to check next.

export ANTHROPIC_API_KEY=...        # or OPENAI_API_KEY, or LOGLENS_API_KEY
python main.py app.log --explain
python main.py app.log --explain --ai-provider openai --ai-model gpt-4o-mini

Without a key the flag prints why it did nothing and the analysis is unchanged. A network failure, a bad key or an unexpected response are all reported the same way: commentary is a bonus, and losing it must never cost the analysis.

Raw log lines are never sent. What goes out is the aggregated brief — counts, signatures, timestamps, health factors — roughly 3 KB, with per-event values masked first (host=db-01 leaves as host=<val>). The masking is the same pass used to build error signatures, so ids, paths, IPs, emails and quoted values are replaced with placeholders.

--ai-base-url is not needed for OpenAI-compatible servers running locally; set LOGLENS_AI_BASE_URL instead (for example http://localhost:11434/v1).

Notifications

--notify posts the result to a webhook. Slack and Discord are recognised from the URL and get a formatted message; anything else receives plain JSON.

python main.py app.log --notify "$SLACK_WEBHOOK"
python main.py app.log --notify "$WEBHOOK" --notify-on anomaly
python main.py logs/ --notify "$WEBHOOK" --notify-on health --min-health 80

--notify-on decides when a run is worth a message: health (default, below --min-health), anomaly, critical, error or always. A run that does not meet the condition says so and sends nothing. A webhook that cannot be reached is reported as a warning — a failed notification never fails the analysis.

Dashboard

--serve starts a local read-only dashboard over the saved history: the latest score, the trend, a table of runs, and a detail panel per run.

python main.py --serve            http://127.0.0.1:8000
python main.py --serve 9000 --open

It is built on http.server rather than a web framework, which keeps the whole tool installable with nothing but Python. The endpoints are plain JSON, so swapping in FastAPI later would not change the contract. The server binds to localhost and only reads.

Docker

The image installs the package and runs the dashboard by default:

docker compose up -d

./logs is mounted read-only at /logs and ./data holds the history database, so saved runs survive a rebuild. Because the entry point is the CLI itself, any command works against the same volumes:

docker compose run --rm dashboard /logs/app.log --save --db /data/loglens.db
docker compose run --rm dashboard /logs/app.log --json --fail-on anomaly

The container runs as an unprivileged user and has a healthcheck against /api/health.

CI

--fail-on controls the exit code: error (default), critical, anomaly, health, or none. Exit 2 means the input could not be read.

python main.py logs/ --json --fail-on anomaly
python main.py logs/ --fail-on health --min-health 80

Layout

loglens/cli.py                  argument parsing and wiring
loglens/parser/formats.py       format rules, timestamp parsing
loglens/parser/log_parser.py    reading, merging, filtering
loglens/analyzer/statistics.py  counting and aggregation
loglens/analyzer/patterns.py    repeats, bursts, clusters
loglens/analyzer/anomalies.py   baseline and deviation
loglens/analyzer/health.py      health score
loglens/analyzer/insights.py    findings in plain language
loglens/analyzer/explain.py     optional LLM commentary
loglens/database/database.py    run history and comparison
loglens/notify.py               webhook notifications
loglens/watcher.py              live following and rolling window
loglens/server.py               dashboard HTTP server
loglens/reports/                console, HTML, JSON, dashboard
tools/make_sample_logs.py       sample data generator
tools/replay_log.py             replays a log to test --watch
tools/make_screenshot.py        renders terminal output as SVG

Everything under loglens/analyzer is pure: it returns data and never prints, so the same code backs the terminal, the HTML report, the JSON output and the dashboard.

Development

pip install -e ".[dev]"
ruff check .
pytest
python tools/make_sample_logs.py

303 tests, no network access from any of them. The sample generator produces a day of traffic containing a deliberate database outage between 18:15 and 19:20, which is what the pattern and anomaly tests are written against.

See CONTRIBUTING.md for the module boundaries and how to add a log format, and docs/KARARLAR.md (Turkish) for why the design decisions were made the way they were.

docs/YOL_HARITASI.md has the detailed roadmap (Turkish).

License

MIT