sys-inspector - eBPF-based System Inspector and Audit Tool

Language / Idioma: English | Português

Sys-Inspector is an advanced observability and forensic tool powered by eBPF (Extended Berkeley Packet Filter).

Unlike traditional tools that poll /proc periodically, Sys-Inspector hooks directly into the Linux Kernel to capture events (process execution, file I/O, network connections) in real-time.

Features (v1.0.0)

• New in v1.0.0 - Answer contract per finding: Every finding declares its confidence (confirmed / probable / heuristic), so a heuristic is never shown as a fact, and its custody (what was preserved of the artifact). The forensic report reads as an investigation: a "how to read" strip (Findings -> Processes -> ATT&CK), a severity legend with the operator action, tooltips on every evidence field, and clickable pivots in both directions between a finding and its ATT&CK technique.

• New in v1.0.0 - Distributed fleet: Pull-model agents forward encrypted captures to a central server (store-and-forward outbox, prioritized ingestion, audited command queue, per-agent capabilities, HTTPS). The Manager shows each command's progress as a live stepper.

• New in v1.0.0 - Runtime and anti-forensic detection: Hidden processes, thread-count divergence, W+X memory, on-disk binary replacement, untrusted libraries, and immutable files in writable directories.

• Forensic Findings: Every collector emits normalized findings on a single severity scale (Info to Critical), each carrying the source that produced it, the MITRE ATT&CK technique, the raw evidence and a recommended action.

• Persistence Enumeration: Answers the first question after a suspected compromise, how would an intruder survive a reboot: systemd units, cron/at jobs, startup and profile scripts, /etc/ld.so.preload, kernel module autoload, udev rules, PAM stacks and per-user authorized_keys. Baseline items stay informational; severity rises only on real indicators such as execution from user-writable paths, world-writable files, hidden names or recent modification.

• Fleet View Dashboard: Monitor multiple infrastructure nodes from a single centralized web interface.

• Forensic Time Machine: Pause live execution and travel back in time to inspect historical snapshots stored in SQLite.

• Kernel-Level Visibility: Uses eBPF kprobes/tracepoints for zero-blindspot monitoring.

• Deep Forensics:

  • Real-time MD5 Hashes: Calculates hashes of executed binaries instantly.
  • Context Awareness: Detects SSH origin IPs, Sudo users, and Tmux sessions.
  • Recursive Alert Bubbling: Child process anomalies (e.g., Unsafe Libs, Net Errors) propagate warnings up to the parent process in the tree view.

• Topology & Infrastructure:

  • Storage Topology: Hierarchical view of Disks -> Partitions -> LVM -> Mount Points with HCTL info.
  • Network Topology: Auto-detection of Gateway, DNS servers, and Interfaces.

• Enterprise Reporting:

  • Generates self-contained, interactive HTML Dashboards.
  • Custom Logo Support: Embeds your organization's logo automatically.
  • Visual Badges: Instant identification of [SSH], [SUDO], [UNSAFE], [NET ERR].
  • Active-state Toolbar: The report toolbar highlights the sort and filter currently applied.

• Dashboard Security (optional):

  • HTTP Basic Authentication: PBKDF2-hashed credentials, working over HTTP and HTTPS.
  • HTTPS with auto self-signed certificate: Zero manual PKI; operator-provided certificates are honored.
  • Both disabled by default (see the "Dashboard Security" section below).

Requirements

• Linux Kernel 4.15+ (5.x+ recommended for BTF support).
• Root privileges (sudo).
• Python 3.6+.
• BCC Tools (python3-bcc).
• iproute2 (for tc command, required only for Chaos Maker).
• Additional Python libs: flask, cryptography, pyyaml.

Installation (PyPI)

Works on any Linux distribution with Python 3.6+.

    pip install sys-inspector

Installation (RPM / openSUSE)

You can install Sys-Inspector directly via zypper using the openSUSE Build Service repository.

1. Add the Repository:

    zypper addrepo https://download.opensuse.org/repositories/home:mariosergiosl:sys-inspector/15.6/home:mariosergiosl:sys-inspector.repo

1. Refresh and Accept GPG Key: During the refresh, you will be asked to trust the repository GPG key.

Fingerprint: 7CF0 5795 053C F397 8E00 948E 9F8D 1AC9 E2BE EABC

    zypper refresh
    # Type 'a' to trust always when prompted.

1. Install the Package:

    zypper install sys-inspector

1. Run: Once installed, the command is available globally:

    sys-inspector

Usage

Sys-Inspector is orchestrated via the main.py entry point (or globally as sys-inspector). It supports multiple execution modes.

1. Local Live Mode (Recommended)

Starts the background collector daemon and the Fleet Web Dashboard simultaneously.

    sudo sys-inspector --mode local-live
    # Access the dashboard at http://localhost:8080

2. Snapshot Mode (Static Report)

Captures activity for a specific duration and generates a standalone HTML report.

    sudo sys-inspector --mode snapshot --interval 20
    # Output Example: report/sys-inspector_hostname_20260316_100000.html

3. Custom Logo

To include your company logo in the report header, simply place a PNG file at the following path:

    /etc/sys-inspector/logo.png

The application will automatically detect, resize (max-height: 40px), encode it to Base64, and embed it in the HTML.

Dashboard Security (Authentication & HTTPS)

Both are optional and disabled by default, so existing deployments are unaffected. Configure them in conf/config.yaml (or /etc/sys-inspector/config.yaml) under the network section.

HTTP Basic Authentication

1. Generate a password hash (run it on the host that serves the dashboard, so the hash matches its werkzeug version):

    python3 tools/gen_password.py

2. Paste the result into config.yaml and enable it:

    network:
      auth:
        enabled: true
        username: "admin"
        password_hash: "pbkdf2:sha256:..."

Authentication works over both HTTP and HTTPS. If enabled without a hash, the server fails closed and rejects all requests.

HTTPS (TLS)

Enable TLS in config.yaml. If the certificate/key below are missing, a self-signed pair is generated automatically on first start (browsers will warn about the unknown issuer, which is expected):

    network:
      tls_enabled: true
      ssl_cert: "/etc/sys-inspector/server_cert.pem"
      ssl_key: "/etc/sys-inspector/server_key.pem"

To use your own PKI, place your certificate and key at the configured paths and they will be used instead of generating one.

See docs/en/dashboard_security.md for details.

Chaos Engineering (Testing Tool)

Included in tools/chaos_maker.sh is a stress testing tool designed to validate the inspector's detection capabilities.

⚠️ WARNING: DO NOT RUN ON PRODUCTION SYSTEMS. This script uses tc (Traffic Control) to purposefully degrade network quality (packet loss/latency) and consumes CPU/Disk resources.

Capabilities

• Network Degradation: Injects 100ms latency and 20% packet loss to trigger [NET ERR] alerts in the report.
• Process Anomalies: Hides processes in /dev/shm to trigger [WARN] alerts.
• Unsafe Library Loading: Forces loading of dynamic libraries from /tmp via a Python script to trigger [UNSAFE] alerts.
• Disk Stress: Generates high I/O throughput to test IO accounting.

How to Run

    sudo ./tools/chaos_maker.sh

To Stop: Press Ctrl+C. The script traps the signal and automatically cleans up the network rules (tc qdisc del) and temporary files.

Project Structure

    ├── conf/                  # Configuration and Cryptographic Keys
    ├── data/                  # SQLite Persistence and Agent IDs
    ├── docs/                  # Narrative documentation (docs/en, docs/pt-BR)
    ├── report/                # Standalone HTML Reports Output
    ├── scripts/               # Development helpers (formatting, venv, test runner)
    ├── src/
    │   ├── collectors/        # eBPF Engine and Process Tree Builders
    │   ├── controllers/       # Execution Modes (Daemon, Web, Snapshot)
    │   ├── core/              # Database and Crypto Logic
    │   ├── exporters/         # HTML and Web Assets
    │   ├── probes/            # C eBPF source code
    │   ├── storage/           # Storage interface and handlers
    │   └── utils/             # Configuration loaders
    ├── tests/                 # Automated test suite (pytest)
    ├── tools/                 # Operational tools (chaos_maker, setup_env, key/password generation)
    └── main.py                # Unified Entry Point

License

Sys-Inspector is free software distributed under the GNU Affero General Public License v3.0 only (AGPL-3.0-only). See LICENSE.md for the full text.

The AGPL was chosen because Sys-Inspector can be operated as a network service (multi-agent server and web dashboard). If you run a modified version and make it available to users over a network, you must offer those users the corresponding source of your modified version.

The license covers the source code only. "Sys-Inspector" and its logo are trademarks and are not licensed with the code; see TRADEMARK.md and NOTICE.