security-shallots 0.2.0

Lightweight self-hosted SIEM — runs on a Raspberry Pi, scales to a server

Security Shallots

A lightweight, self-hosted SIEM that runs on anything — from a Raspberry Pi to a full server.

No Splunk. No Elastic. No Docker required. Just Python and SQLite.

pip install .
shallot run

Open http://localhost:8844 and you're monitoring.

What it does

Security Shallots collects security alerts from your network, deduplicates and enriches them, detects attack patterns, and gives you a single dashboard to review everything.

Sources it can ingest:

• Suricata IDS (EVE JSON logs)
• Syslog (routers, firewalls, anything that speaks syslog)
• pfSense (firewall logs + DHCP via API)
• Wazuh (HIDS/EDR alerts)
• CrowdSec (behavioral blocks)
• Pi-hole (DNS query logs)
• Argus endpoint agents (Windows EDR)
• Web application logs

What it does with them:

• Normalizes alerts into a common format
• Deduplicates repeated noise
• Auto-classifies severity and suppresses known false positives
• Correlates patterns (port scans, brute force, lateral movement)
• Groups related alerts into incidents with response runbooks
• Optional AI triage via API (OpenAI/Anthropic) — no local GPU needed
• Web dashboard with real-time WebSocket feed

Hardware auto-detection

Shallots detects your hardware on startup and adjusts automatically:

System	RAM	CPU	What runs
Raspberry Pi	2 GB	1 core	Suricata + syslog, basic correlations
Old laptop	4 GB	2 cores	+ CrowdSec, incidents, AI via API
Desktop	8 GB	4 cores	+ Wazuh, agents, full analytics
Server	16+ GB	8+ cores	Full threat engine (baselines, graph, ML, kill chain)

On a 4 GB machine, Shallots uses ~200-400 MB RAM with the threat engine disabled. Storage: 30 days of alerts from a home network uses ~200-500 MB of SQLite.

Quick start

Option A: One-line installer (recommended)

Installs everything — Python, Suricata, Shallots — detects your hardware, and starts monitoring:

curl -fsSL https://raw.githubusercontent.com/benolenick/security-shallots/main/setup/shallot-setup | sudo bash

The installer walks you through profile selection, network config, and component choices. At the end you get a running dashboard and agent deploy commands.

Option B: Manual install

# Requires Python 3.10+ and Suricata
pip install git+https://github.com/benolenick/security-shallots.git
shallot setup                    # auto-detect network, generate config
shallot setup --install-service  # also install systemd service
shallot run                      # start (foreground)

Key settings:

• network.home_cidr — your LAN range (e.g. 192.168.1.0/24)
• suricata.eve_path — path to Suricata's eve.json
• web.username / web.password — dashboard login (recommended)
• profile — auto (default), or force lite / micro / standard / full

3. Run

# Foreground (for testing)
shallot run

# As a background service (if you used --install-service)
sudo systemctl enable --now shallotd

4. Dashboard

Open http://your-ip:8844.

5. Deploy agents (optional)

Install agents on your machines to monitor logins, file changes, processes, and more. The dashboard has a guided setup (click Setup in the header), or use the command line:

# Linux — installs Wazuh agent + clove-watchdog (~15 MB, runs as systemd service)
curl -fsSL https://raw.githubusercontent.com/benolenick/security-shallots/main/setup/endpoint/clove \
  | sudo bash -s -- --manager YOUR_SERVER_IP

# Windows — installs Argus sentinel (auto-installs Python if needed, runs as scheduled task)
irm https://raw.githubusercontent.com/benolenick/security-shallots/main/setup/endpoint/clove.ps1 | iex

Agents auto-register and start sending alerts within minutes. Uninstall with --uninstall (Linux) or -Uninstall (Windows).

Note: Linux agents require Wazuh Manager on the server (enabled in standard and full profiles). On lite/micro profiles, use syslog forwarding instead — the dashboard Setup tab shows how.

Profiles

Set profile: auto (the default) and Shallots picks the right mode:

Profile	RAM	Components	Threat engine
lite	< 2 GB	Suricata + syslog	Off
micro	2-4 GB	+ CrowdSec	Off
standard	4-8 GB	+ Wazuh, VictoriaLogs, Grafana	Tuned down
full	8+ GB	Everything including Argus agents	Full

Override: profile: lite in config.yaml.

AI triage

Works without AI — you can manually review alerts. Add an API key for automatic classification:

ai:
  tier: remote_api
  anthropic_api_key: "sk-ant-..."   # or openai_api_key

Cost: a few cents per day for a typical home network. No GPU needed.

For local AI (if you have the hardware):

ai:
  tier: local
  ollama_url: "http://localhost:11434"
  ollama_model: "qwen3:14b"

pfSense integration

Configure pfSense to send syslog to your Shallots host:

• Status > System Logs > Settings > Remote Logging > add your Shallots IP
• Enable syslog in config.yaml with matching port

Optional API integration pulls DHCP leases for IP-to-hostname mapping.

CLI

shallot run                              # Start daemon (foreground)
shallot status                           # Component health + alert stats
shallot query "show SSH attacks today"   # AI natural language query
shallot health                           # Detailed health check

Architecture

[Suricata] ──┐
[Syslog]   ──┤
[pfSense]  ──┼──> Queue ──> Pipeline ──> SQLite ──> REST API ──> Dashboard
[Wazuh]    ──┤              (normalize,            (aiohttp)     (vanilla JS)
[CrowdSec] ──┘               dedup,
                              enrich,
                              classify)
                                |
                          Correlator ──> Incidents
                          (port scan,    (grouped alerts
                           brute force,   with runbooks)
                           lateral mvmt)

• Python asyncio — single process, cooperative multitasking
• SQLite + FTS5 — full-text search, no external database
• 5 pip dependencies — aiohttp, aiosqlite, pyyaml, maxminddb, aiofiles
• Vanilla JS frontend — no React, no build tools, no node_modules

Project structure

shallots/
  daemon.py          # Main orchestrator — starts all workers
  config.py          # YAML config + hardware auto-detection
  cli.py             # CLI entry point
  ingest/            # Data source ingestors (suricata, syslog, etc.)
  pipeline/          # normalize -> dedup -> enrich -> classify
  ai/                # Triage, correlator, incidents, autopilot
  store/             # SQLite database layer
  web/
    api/             # REST API endpoints (split by domain)
    app.py           # aiohttp app factory
    static/          # Dashboard (vanilla JS SPA)

Development

python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install -e ".[dev]"
python -m shallots run --debug

License

MIT