Table of Contents

• Overview
• Why Spectre?
• Architecture
• Key Features
• Getting Started
  • Prerequisites
  • Installation
• Usage & CLI Reference
• Detection Rules & Sigma Integration
• Testing & Verification
• Documentation & Roadmap
• Acknowledgments & External Links

Overview

Project Spectre is a lightweight, local-first behavioral Host Intrusion Detection System (HIDS). Rather than relying heavily on static file signatures, Spectre models the grammar of host processes and resource actions to detect anomalous execution chains and behaviors.

Currently on V10 (Active Containment) with Phase 1 (Detection Engineering Platform) features, Spectre tracks process lineages, monitors file and network I/O, evaluates threats in real-time, provides MITRE ATT&CK context, scans payloads via YARA, and can actively quarantine or terminate malicious process trees.

New in Phase 1:

• Sigma-native rule format — Write rules in Sigma YAML, convert to Spectre automatically
• 24 built-in rules across 4 packs: webshell, privilege escalation, credential access, lateral movement
• Rule testing CLI — Validate Sigma syntax, test conversion, replay traces
• Rule pack management — Install/uninstall community rule packs
• Detection-as-code ready — CI/CD integration for rule validation

Why Spectre?

Traditional endpoint protection platforms (EPP) and antiviruses (AV) often rely heavily on static signatures—checking file hashes against a known database of malware. This approach completely fails against zero-day threats, fileless malware, and "living off the land" techniques where attackers abuse legitimate system binaries (like powershell, curl, or bash).

Spectre shifts the security paradigm from static characteristics to dynamic relationships. By continuously tracking process ancestry (who spawned who) and correlating it with resource access (who touched which file, who opened which socket), Spectre identifies malicious intent rather than malicious files.

Example Attack Chain Detected:

nginx (Web Server)
└── bash (Interactive Shell)
    ├── curl (Downloads payload)
    │   └── [WRITE] -> /tmp/malware.sh
    └── sh (Executes payload)
        └── [CONNECT] -> 192.168.1.50:4444 (C2 Server)

Architecture

The system operates across a 4-stage pipeline:

+---------------------+      +---------------------+      +---------------------+
|                     |      |                     |      |                     |
|  1. OS Telemetry    |----->|  2. Graph Builder   |----->| 3. Detection Engine |
|  (psutil / eBPF)    |      |  (NetworkX Memory)  |      |  (Rules & Scoring)  |
|                     |      |                     |      |                     |
+---------------------+      +---------------------+      +---------------------+
                                                                     |
                                                                     v
                                                          +---------------------+
                                                          |                     |
                                                          |  4. Action & Alert  |
                                                          | (Containment, REST) |
                                                          |                     |
                                                          +---------------------+

1. Telemetry Sensing: Continuously polls the OS for process spawns, file descriptors, and network sockets (psutil) — eBPF sensor in development for zero-gap visibility.
2. Graph Construction: Events are normalized into a sliding-window, directed process-resource graph, automatically pruning stale events to prevent memory leaks.
3. Detection Engine: The active graph is evaluated against JSON/Sigma-configurable behavioral rules. Threat scores accumulate along process lineage chains.
4. Action & Visualization: Once a threshold is breached, Spectre fires an alert, maps it to MITRE ATT&CK, runs a deep-scan via YARA, and can actively freeze/kill the process tree. REST API + Next.js dashboard for real-time monitoring.

Key Features

• Process Ancestry Tracking: Reconstructs complete execution lineages, handling PID recycling and short-lived processes safely.
• Resource Monitoring: Tracks I/O operations including READ/WRITE for files, and CONNECT/LISTEN for sockets.
• Behavioral Detection Engine: Scores chains of events dynamically using JSON-configurable rules.
• Sigma-Native Rules: Write rules in Sigma YAML; automatic conversion to Spectre format with MITRE ATT&CK extraction.
• Threat Enrichment:
  • MITRE ATT&CK: Alerts mapped automatically to ATT&CK tactics (e.g., T1059 - Command and Scripting Interpreter).
  • YARA Integration: Scans suspicious files on-the-fly using the yara-python engine.
• Active Containment: Configurable actions (--contain stop or kill) to instantly freeze or terminate entire threat process trees.
• Persistence & API: Events and alerts stored in local SQLite database, exposed via FastAPI REST interface.
• Live Dashboard: Real-time web dashboard for monitoring graph, alerts, and system telemetry.
• Detection Engineering Platform: Rule testing, Sigma validation, pack management, CI/CD integration.

Getting Started

Prerequisites

• Python 3.8 or newer
• Linux Operating System (for accurate /proc mapping and psutil compatibility)
• Root privileges (required for process monitoring and containment)
• Dependencies: psutil, networkx, fastapi, yara-python, uvicorn, click, rich, pyyaml

Installation

Option 1: One-line installer (recommended)

curl -sSL https://raw.githubusercontent.com/Aayushbankar/spectre/main/install.sh | sudo bash

Option 2: PyPI

pip install spectre-hids[yara]

Option 3: Docker

docker run -d --privileged --pid=host --cgroupns=host \
  -v /:/host:ro \
  -v /var/log/spectre:/var/log/spectre \
  -v /var/lib/spectre:/var/lib/spectre \
  ghcr.io/aayushbankar/spectre:latest \
  --contain kill --api

Option 4: From source

git clone https://github.com/Aayushbankar/spectre.git
cd spectre
pip install -e ".[yara]"

Usage & CLI Reference

Spectre provides a highly configurable Command Line Interface (CLI) for tuning the engine's sensitivity and enabling specific modules.

Basic Usage

Run with Active Containment & REST API:

spectre run --verbose --contain kill --api

Full CLI Commands

spectre run Options

Detection Rules & Sigma Integration

Built-in Rule Packs (24 Sigma Rules)

Install Rule Packs

# List available packs
spectre rule list

# Install webshell detection pack
spectre rule install webshell

# Install all packs
for p in webshell privilege_escalation credential_access lateral_movement; do
  spectre rule install "$p"
done

Write Custom Sigma Rules

title: Suspicious Python Script Execution
id: custom-suspicious-python-001
description: Detects python executing scripts from /tmp or /dev/shm
status: experimental
author: Your Name
date: 2024-01-15
logsource:
    category: process_creation
    product: linux
detection:
    selection_python:
        Image|endswith: '/python3'
    selection_tmp_script:
        CommandLine|contains:
            - '/tmp/'
            - '/dev/shm/'
    condition: selection_python and selection_tmp_script
level: high
tags:
    - attack.t1059.006
    - attack.execution

Test Rules

# Validate Sigma syntax and test conversion
spectre test rule my_rule.yml

# Test all rules in a pack
spectre test rule spectre/rules/packs/webshell/*.yml

Testing & Verification

Spectre includes an automated E2E verification test suite to simulate and assert threat escalation behaviors.

Run V10 Test Suite

python -m pytest tests/v10/run_test.py

Run Unit Tests

python -m pytest tests/unit/ -v

Run Rule Tests (CI)

# Validates Sigma syntax and conversion
python -m pytest tests/ -k "sigma" -v

Manual Verification

# Start Spectre in background with low threshold
spectre run --threshold 5 --contain kill --api &

# Trigger a test alert (reads /etc/hosts with python)
python -c "open('/etc/hosts').read(); import time; time.sleep(10)"

# Check dashboard at http://localhost:8000

Documentation & Roadmap

Detailed architectural notes and version progression can be found in the docs/ directory:

• Master Design Document: Vision, entity relations, and the 17-stage SDLC roadmap.
• Progress Tracker: Current completion status of the project.
• GTU Internship Submission: Details for project submission.

Incremental SDLC Roadmap (Current Status):

• V0-V3: Process Monitor, Rule Engine, Resource Tracking, Graph Memory.
• V4-V5: Detection Engine, MITRE ATT&CK Mapping.
• V6-V8: SQLite Persistence, REST API, Live Dashboard.
• V9: YARA Engine Integration.
• V10: Active Containment (SIGSTOP/SIGKILL).
• Phase 1: Sigma Integration, Rule Packs, Testing CLI, Pack Management.
• V11: Attack Replay Framework (Atomic Red Team).
• V12: OS Telemetry Upgrades (eBPF, auditd).
• V13-V17: Machine Learning, Graph Embeddings, Multi-host Agent.

Acknowledgments & External Links

Spectre is built on the shoulders of giants. We heavily rely on the following open-source frameworks and security standards:

• psutil: For cross-platform OS-level process and system monitoring.
• NetworkX: For sliding-window directed graph processing and ancestry modeling.
• FastAPI: For exposing the high-performance telemetry API.
• YARA: The pattern matching swiss knife for malware researchers.
• MITRE ATT&CK®: The globally-accessible knowledge base of adversary tactics and techniques.
• Sigma: Generic signature format for SIEM systems.
• Atomic Red Team: Atomic tests for ATT&CK techniques.