logowatch 0.4.0

Autonomous log analysis tool with configurable pattern rules, incremental detection, and rich output

Logowatch

Autonomous log analysis tool with configurable pattern rules, incremental detection, and rich output.

Features

• Pattern-based rules - Define regex patterns to match in log files
• Incremental detection - Only show new errors since last check
• Extended options - Context, value extraction, aggregations
• Multiple sources - Files, directories, glob patterns
• Rich output - Colored CLI output with tables and panels
• JSON output - Machine-readable output for automation
• Legacy support - Convert from pipe-delimited format

Installation

pip install logowatch

Quick Start

1. Create a configuration file

logowatch init

This creates a logowatch.yaml file you can customize.

2. Run analysis

logowatch analyze logowatch.yaml

Quick scan a single file

logowatch scan /var/log/app.log --pattern "ERROR" --pattern "Exception"

Configuration

YAML Format

# logowatch.yaml

sources:
  - name: app
    path: /var/log/app.log
    type: file

  - name: errors
    path: /var/log/app/errors/
    type: directory
    pattern: "*.log"

rules:
  # Basic rule
  - pattern: "ERROR|CRITICAL"
    description: "Application errors"
    severity: error
    source: app

  # With context (show lines after match)
  - pattern: "Traceback"
    description: "Python exceptions"
    severity: error
    source: app
    options:
      context:
        after: 10

  # Extract values
  - pattern: "user_id=\\d+"
    description: "Errors with user IDs"
    severity: error
    source: app
    options:
      extract:
        pattern: "user_id=(\\d+)"
        name: "user_id"

  # Aggregate by extracted value
  - pattern: "status_code=\\d+"
    description: "HTTP status codes"
    severity: warning
    source: app
    options:
      extract:
        pattern: "status_code=(\\d+)"
        name: "status"
      aggregate:
        by: "(\\d{3})"
        format: count

  # Case-insensitive matching
  - pattern: "timeout|connection refused"
    description: "Connection issues"
    severity: warning
    source: app
    case_insensitive: true

Rule Options

Option	Description
context.before	Lines to show before match
context.after	Lines to show after match
extract.pattern	Regex with capture group
extract.name	Name for extracted value
aggregate.by	Regex for grouping
find_related.pattern	Related pattern to find
find_related.lines	Lines to search
find_related.direction	before/after/both

Severity Levels

• error - Critical issues (shown in red)
• warning - Warnings (shown in yellow)
• info - Informational (shown in cyan)

CLI Commands

analyze

Run full analysis with configuration file:

logowatch analyze config.yaml [OPTIONS]

Options:
  -l, --limit INTEGER      Max examples per rule (default: 5)
  --no-incremental         Show all errors (not just new)
  -s, --source TEXT        Analyze specific source(s)
  --json                   Output as JSON
  -c, --cache-path PATH    Custom cache file path

scan

Quick scan a single file:

logowatch scan logfile.log [OPTIONS]

Options:
  -p, --pattern TEXT    Pattern(s) to search for
  -l, --limit INTEGER   Max matches (default: 10)

convert

Convert legacy pipe-delimited config to YAML:

logowatch convert rules.conf -o logowatch.yaml

init

Create sample configuration:

logowatch init

Incremental Mode

By default, logowatch tracks which lines it has already processed and only shows new errors since the last check. This is perfect for daily monitoring.

Cache is stored in .logowatch_cache.json by default.

To show all errors (ignore cache):

logowatch analyze config.yaml --no-incremental

Python API

from logowatch import LogAnalyzer, load_config

# Load configuration
config = load_config("logowatch.yaml")

# Create analyzer
analyzer = LogAnalyzer(config, incremental=True)

# Run analysis
result = analyzer.analyze()

# Check results
print(f"Errors: {result.total_errors}")
print(f"Warnings: {result.total_warnings}")

for rule_result in result.rule_results:
    print(f"{rule_result.rule.description}: {rule_result.total_count}")
    for match in rule_result.matches[:3]:
        print(f"  {match.content}")

Legacy Format Support

If you have existing rules in pipe-delimited format:

# pattern|description|severity|show_source|case_insensitive|source|options
ERROR|Application errors|ERROR|true|false|app
Timeout|Timeout issues|WARNING|false|true|app|{"context": {"after": 3}}

Convert to YAML:

logowatch convert rules.conf -o logowatch.yaml

License

MIT

Author

LifeAiTools Team [e-mail]