log-compactor 0.1.0

A real-time, configurable log deduplication and compaction tool.

Log Compactor

A real-time, configurable log deduplication and compaction utility. It supports dynamic regex parsing, key-value extraction, raw pass-through for unstructured lines, time-window deduplication, and error escalation.

Installation

From the project root (development):

pip install -e ".[dev]"

Or install the package only:

pip install -e .

The dev extra includes pytest.

Pipe any command’s output

Install the package, add a config.yaml (or pass --config), and pipe stdout through the CLI:

pytest -q 2>&1 | logcompact stream -c /path/to/config.yaml

python app.py 2>&1 | logcompact stream

If you omit -c, the CLI looks for config.yaml in the current working directory.

Library usage

from compactor import SmartCompactor

settings = {"dedup_window_seconds": 5, "error_threshold": 2}
compactor = SmartCompactor(settings)
for line in compactor.compact_stream(open("app.log")):
    print(line)

Tests

pytest

Changelog

See CHANGELOG.md for version history and release notes.

License

This project is licensed under the GNU General Public License v3.0 only. See the LICENSE file for the full license text.