snap-analog 2.1.0

High-performance log analysis and visualization toolkit

📊 snap-analog

Advanced Log Analysis & Visualization Toolkit for Apache Logs

snap-analog is a powerful command-line toolkit for parsing, analyzing, optimizing, and visualizing large-scale Apache log files. It generates both beautiful visual dashboards and structured JSON datasets, making it perfect for DevOps monitoring, security analysis, and data analytics pipelines.

📑 Quick Links

⚡ Quick Start • 🚀 Features • 🎯 Use Cases • 📦 Installation • 🧪 Usage • 📊 JSON Output • 💡 Performance • 🎨 Visualization Features • 🧑‍💻 Development • 🤝 Contributing • 📜 License • 🌟 Roadmap • 📞 Support & Contact • ⭐ Show Your Support

---

⚡ Quick Start

# Install
git clone https://github.com/batuhannerkoc/snap-analog.git
cd snap-analog
pip install .

# Generate test data
snap-analog generate-test --lines 5000 --output test.log

# Analyze & visualize
snap-analog analyze test.log --visualize

# View dashboard in: reports/dashboard_*.png

That's it! 🎉

---

📸 Screenshots

Dashboard Visualization

Terminal Output

---

🚀 Features

• 🔍 High-performance log parsing — optimized for millions of lines
• 💾 Memory-optimized modes — auto, balanced, full, aggressive
• 📈 JSON export — structured data for analytics pipelines and dashboards
• 🎨 Beautiful visualizations — powered by Matplotlib + Seaborn
• 🧠 Traffic insights — top IPs, URLs, methods, status groups, time-series
• ⚠️ Error rate detection — automatic threshold alerts
• 🧪 Test log generator — built-in random data generation
• 🛠 Modern CLI — colorful output with progress bars
• 📊 6 chart types — pie charts, bar charts, time-series, heatmaps

---

🎯 Use Cases

Use Case	Description
🖥 Web Server Monitoring	Track traffic patterns, identify bottlenecks
🔒 Security Analysis	Detect suspicious IPs, analyze attack patterns
📊 SRE/DevOps Dashboards	Feed data into Grafana, Kibana, or custom tools
🚀 API Performance Tracking	Monitor endpoint response times and error rates
📉 Data Analytics	Preprocess logs for ML pipelines
🧪 Performance Testing	Generate realistic test data at scale

---

📦 Installation

✔ Option 1 — Virtual Environment (Recommended)

git clone https://github.com/batuhannerkoc/snap-analog.git
cd snap-analog

python3 -m venv venv
source venv/bin/activate  # macOS/Linux
# .\venv\Scripts\activate # Windows

pip install -r requirements.txt
pip install .

Verify installation:

snap-analog --help

✔ Option 2 — Direct Install

pip install .

If you encounter "externally managed environment" error:

pip install --user .

✔ Option 3 — Development Mode

pip install -e .

---

🧪 Usage Examples

Generating Test Logs

# Apache format
snap-analog generate-test --lines 5000 --output logs/test.log --format apache

# JSON format
snap-analog generate-test --format json --lines 3000 --output logs/sample.json

Analyzing Logs

# Basic analysis
snap-analog analyze access.log

# With visualization
snap-analog analyze access.log --visualize

# Aggressive memory mode (for huge files)
snap-analog analyze access.log --mode aggressive

# Custom output path
snap-analog analyze access.log --output reports/result.json

# Quiet mode (no terminal output)
snap-analog analyze access.log --quiet

Visualizing Reports

# Basic visualization
snap-analog visualize reports/log_analysis_20250101_120000.json

# Custom theme and size
snap-analog visualize report.json --theme darkgrid --size large --dpi 200

---

📊 JSON Output Example

{
  "summary": {
    "total_lines": 50214,
    "total_requests": 48711,
    "memory_mode": "balanced",
    "file": "access.log",
    "analysis_date": "2025-01-12T10:30:45"
  },
  "health_metrics": {
    "success_rate_2xx_3xx": "94.1%",
    "client_error_rate_4xx": "3.2%",
    "server_error_rate_5xx": "2.7%"
  },
  "traffic_analysis": {
    "top_ips": [
      {"ip": "192.168.1.1", "count": 1250},
      {"ip": "10.0.0.5", "count": 980}
    ],
    "top_urls": [
      {"url": "/api/users", "count": 5420},
      {"url": "/home", "count": 3210}
    ],
    "methods": {
      "GET": 35420,
      "POST": 8940,
      "PUT": 2130,
      "DELETE": 1221
    }
  },
  "elapsed_time": 3.51
}

📄 Full Schema: View complete JSON structure

---

🏗 Project Structure

snap-analog/
│
├── src/
│   ├── cli.py                # Main CLI entry point
│   ├── log_analyzer.py       # Memory-optimized analyzer
│   └── log_visualizer.py     # Dashboard generator
│
├── images/
│   ├── dashboard.png         # Dashboard screenshot
│   └── terminal.png          # Terminal UI screenshot
│
├── sample_logs/                     # Sample Log files
├── requirements.txt
├── setup.py
├── LICENSE
└── README.md

---

⚙️ Requirements

matplotlib>=3.5.0
seaborn>=0.12.0
pandas>=1.4.0
numpy>=1.22.0
psutil>=5.9.0  # optional, for system info

Install all dependencies:

pip install -r requirements.txt

---

💡 Performance Benchmarks

Tested on: MacBook Pro M4, 16GB RAM

Log Size	Lines	Mode	Time	Memory
Small	10K	auto	0.5s	45 MB
Medium	100K	balanced	2.1s	120 MB
Large	1M	balanced	8.4s	380 MB
Huge	10M	aggressive	42s	850 MB

---

🎨 Visualization Features

The dashboard includes:

1. Status Groups Pie Chart — 2xx, 3xx, 4xx, 5xx distribution
2. Top IPs Bar Chart — Most active IP addresses
3. Top URLs Bar Chart — Most requested endpoints
4. Time-Series Traffic — Requests over time
5. HTTP Methods Distribution — GET, POST, PUT, DELETE breakdown
6. Error Rate Heatmap — Visual error rate indicators

All visualizations support:

• Custom themes (white, dark, darkgrid, whitegrid)
• Adjustable DPI (72, 100, 150, 200, 300)
• Multiple sizes (small, medium, large, xlarge)

---

🧑‍💻 Development

Editable Install

git clone https://github.com/batuhannerkoc/snap-analog.git
cd snap-analog
pip install -e .

Run CLI Directly

python3 src/cli.py analyze logs/test.log

Code Style

This project follows Black code style:

pip install black
black src/

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

For bug reports or feature requests, please open an issue.

---

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

MIT License

Copyright (c) 2025 Batuhan Erkoc

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

---

🌟 Roadmap

• Support for Nginx logs
• Real-time monitoring mode (snap-analog watch)
• HTML report generation
• CSV export option
• Docker image
• PyPI package (pip install snap-analog)
• Filtering capabilities (--filter "status=500")
• Log comparison tool

---

📞 Support & Contact

• GitHub Issues: Report bugs or request features
• Email: [e-mail]
• LinkedIn: Batuhan Erkoc

---

⭐ Show Your Support

If you find this project useful, please consider:

• ⭐ Starring the repository
• 🐛 Reporting bugs
• 💡 Suggesting new features
• 🔀 Contributing code

---

Developed with ❤️ by Batuhan Erkoc