TraceForge

Open-source OSINT, DFIR and security investigation toolkit.

---

What It Does

TraceForge is an open-source command-line toolkit for OSINT, digital forensics, security investigation, evidence handling, and first-party investigation tools:

• First-Party Native Tools (traceforge / go/): High-performance Go-native and Python utilities providing asset relationship graphing, snapshot diffing, streaming IOC extraction/defanging, forensic evidence indexing, UTC timeline normalization, PCAP network flow analysis, automated log triage, and filesystem baselining (see docs/FIRST_PARTY_TOOLS.md).
• Tool Catalog & Installer: A searchable catalog of 152 tools and an automated installer that handles dependencies across Homebrew (macOS), APT (Linux), pipx (Python virtual environments), Go, Cargo, and RubyGems.
• Investigation Modules: 7 built-in analysis scripts covering image metadata, network PCAPs, usernames, email breach records, domain DNS records, document metadata, and local OPSEC hygiene.
• Case Management: Workspaces stored under workspace/ with unique case IDs, SHA-256 evidence hashing, and append-only chain-of-custody logging.
• Export Pipeline: Generates reports in Markdown, standalone dark-mode HTML, CSV (with formula injection protection), TSV, JSON, JSONL streams, STIX 2.1, MISP, GeoJSON, KML, Excel XLSX, Word DOCX, and signed ZIP packages.
• Redaction: Built-in --redact flag masks sensitive IP addresses and email addresses in generated reports.

---

Architecture

TraceForge does not ship binary copies of third-party tools inside the repository. It installs and launches external tools via your system's package managers:

+-------------------------------------------------------------------+
|                            TraceForge                             |
|      (CLI Menu, Tool Catalog, Case Workspaces, Report Exporter)   |
|                         [ MIT License ]                           |
+---------------------------------+---------------------------------+
                                  |
                                  | Installs / Launches / Collects Output
                                  v
+-------------------------------------------------------------------+
|                 Third-Party Command-Line Tools                    |
|   (ExifTool, TShark, Binwalk, Nmap, Sherlock, Maigret, etc.)     |
|         [ Retain Respective Upstream Copyright & Licenses ]        |
+-------------------------------------------------------------------+

All third-party tools remain the property of their respective creators and are licensed under their own open-source terms. See THIRD_PARTY_NOTICES.md for details.

---

Supported Systems

Platform	Tier	Architecture	Package Manager	Guide & Notes
macOS Apple Silicon	Supported	arm64 (M1/M2/M3/M4)	Homebrew	Full native support
macOS Intel	Supported	x86_64	Homebrew	Full native support
Debian / Ubuntu	Supported	x86_64, arm64	APT	Full native support
Kali Linux	Supported	x86_64, arm64	APT	Full native support
Termux / Android	Supported with limitations	arm64, armv7, x86_64	pkg	Non-root userland. Offline forensics & PCAP analysis. Termux Guide

---

Installation & Quick Start

1. Clone the Repository

git clone https://github.com/paman7647/TraceForge.git
cd TraceForge
chmod +x install_all.sh main.sh modules/*.sh scripts/*.sh tests/*.sh

2. Run the Installer

# Preview packages without installing (dry-run)
./install_all.sh --profile recommended --dry-run

# Core Profile: Minimal tools for the 7 modules (~15 packages)
./install_all.sh --profile core

# Recommended Profile (Default): Standard investigative toolset (~50 packages)
./install_all.sh --profile recommended

# Full Profile: All 152 installable catalog tools
./install_all.sh --profile full

3. Run TraceForge

# Interactive TTY Menu
./main.sh
# or via Python CLI
python3 -m traceforge

# Run system diagnostics
./main.sh doctor

# List existing cases
./main.sh list-cases

# Search the catalog
./main.sh search "steganography"

# Run first-party tools
traceforge ioc extract /path/to/intel.txt
traceforge asset graph /path/to/entities.jsonl --html graph.html

# Run a module directly
./main.sh module 1 /path/to/evidence.png CASE-20260825-ABC123

# Export a case
./main.sh export CASE-20260825-ABC123 --all --package zip

---

Investigation Modules

Module	Script	Target & Tools Used
01. Media & Image Forensics	modules/01_image_forensics.sh	Images and videos: EXIF/IPTC metadata, GPS coordinates, strings, steganography (exiftool, binwalk, zsteg, xxd).
02. Network & PCAP Forensics	modules/02_network_recon.sh	Packet captures: DNS queries, HTTP URIs, TLS SNI headers, conversations (tshark, capinfos, aircrack-ng).
03. Identity & Social Research	modules/03_identity_social.sh	Usernames and aliases: account discovery across web platforms (sherlock, maigret, blackbird, socialscan).
04. Email & Breach Intelligence	modules/04_email_breach.sh	Email addresses: account registrations, breach dumps, SPF/DMARC posture (holehe, h8mail, emailrep, theHarvester, checkdmarc).
05. Domain & DNS Intelligence	modules/05_domain_dns.sh	Domains: DNS records, passive subdomains, HTTP probing, typosquats (dig, whois, subfinder, amass, dnsx, httpx, dnstwist).
06. Document & Metadata Harvesting	modules/06_document_harvesting.sh	Documents (PDF, DOCX, XLSX): properties, text, embedded objects, macros, secret keys (poppler, oletools, exiftool, qpdf, ripgrep).
07. OPSEC & Environment Audit	modules/07_opsec_anonymization.sh	Local audit: public IP detection, DNS leak test, Tor/proxy status, encryption tools (mat2, tor, proxychains, gnupg, age).

---

Directory Structure

TraceForge/
├── main.sh                        # Interactive console and CLI dispatcher
├── install_all.sh                 # Multi-ecosystem package installer
├── pyproject.toml                 # Modern Python package configuration
├── go.mod                         # Go module declaration (zero external dependencies)
├── VERSION                        # Project version (1.0.0)
├── LICENSE                        # MIT License
├── NOTICE                         # Copyright and attribution
├── THIRD_PARTY_NOTICES.md         # Upstream tool licenses and repository links
├── README.md                      # Main project documentation
├── SECURITY.md                    # Vulnerability reporting guidelines
├── CONTRIBUTING.md                # Contribution guidelines
├── CODE_OF_CONDUCT.md             # Contributor Covenant 2.1
├── CHANGELOG.md                   # Version release history
├── RELEASE_NOTES.md               # 1.0.0 release notes
│
├── go/                            # Go-native high-performance engine
├── traceforge/                    # Pure Python implementation and package
│
├── catalog/
│   ├── README.md                  # Catalog schema specification
│   ├── tools.tsv                  # 15-column registry of 152 tools
│   └── TOOLS.md                   # Formatted markdown tool index
│
├── lib/
│   ├── common.sh                  # Terminal UI, colors, logging, string helpers
│   ├── platform.sh                # OS and architecture detection
│   ├── packages.sh                # Package managers (Homebrew, APT, pipx, Go, Gem, Cargo)
│   ├── catalog.sh                 # Catalog search and filter functions
│   ├── case.sh                    # Case lifecycle, evidence, findings, IOCs, timeline
│   ├── export.sh                  # Export coordinator, packaging, and hashing
│   └── report.sh                  # Markdown, HTML, and PDF report generators
│
├── modules/                       # 7 investigation modules (01 to 07)
├── scripts/
│   ├── doctor.sh                  # System and dependency diagnostic checker
│   ├── export_case.sh             # CLI export wrapper
│   ├── export_engine.py           # Multi-format export engine
│   ├── install_tool.sh            # Single tool installer
│   ├── generate_catalog_docs.sh   # Markdown documentation generator
│   └── release_check.sh           # Pre-flight release verification test
│
├── docs/                          # In-depth operator, architecture, and OPSEC guides
├── tests/                         # Automated test suite and regression tests
└── workspace/                     # Local investigation cases
    └── .gitkeep

---

Development & Branching Workflow

TraceForge maintains two long-lived branches:

• master: Stable, production-ready public release branch.
• beta: Active integration branch for new features, bug fixes, platform improvements, and release candidates.

                  master  (Stable Releases)
                    ▲
                    │  Release PR (Validated & Tagged)
                    │
                   beta   (Active Development & RC)
                ▲   ▲   ▲
                │   │   │  Pull Requests
           feature fix docs  (Short-Lived Working Branches)

Pull requests and contributions must target the beta branch. See docs/BRANCHING.md for the branching model, docs/BUG_REPORTING.md for issue guidelines, and CONTRIBUTING.md for pull request workflows.

---

Automated Testing

Run the test suite before submitting pull requests or publishing releases:

# Run Python & Platform test suites
python3 -m unittest discover -s tests

# Run master shell & integration test harness
./tests/test.sh

# Run pre-flight release check
./scripts/release_check.sh beta

All shell scripts are validated with bash -n and shellcheck -x.

---

Responsible Use

TraceForge is intended for lawful OSINT, digital forensics, incident response, security research, authorized testing, education, and lab work.

You are responsible for ensuring that your use of the software is authorized and complies with applicable law, regulation, contracts, platform terms, privacy requirements, and organizational policy.

TraceForge does not grant permission to access any system, account, network, service, or data.

See DISCLAIMER.md and RESPONSIBLE_USE.md for details.

---

Documentation & Policies

• docs/BRANCHING.md — Two-branch Git model, release lifecycle, and PR rules.
• docs/BUG_REPORTING.md — Bug reporting protocol, reproduction requirements, and fix lifecycle.
• docs/RUNTIME_PROFILES.md — Architecture and configuration of the 6 runtime profiles.
• docs/FIRST_PARTY_TOOLS.md — Documentation for first-party analytical tools.
• docs/platforms/termux.md — Termux and Android platform guide and storage configuration.
• DISCLAIMER.md — Legal disclaimer, liability boundaries, accuracy notices, and jurisdictional advice.
• RESPONSIBLE_USE.md — Responsible use standards, authorized scope, and prohibited activities.
• PRIVACY.md — Privacy principles, local storage layout, data minimization, and redaction.
• SECURITY.md — Vulnerability reporting policy and defensive engineering standards.
• THIRD_PARTY_NOTICES.md — Upstream tool licenses, attribution, and license boundaries.
• CONTRIBUTING.md — Contribution workflow, code formatting, and review guidelines.
• LICENSE — Official MIT License terms and conditions.

---

Authors & Credits

• Lead Architect & Core Developer: Aman Kumar Pandey
• Credits: Thanks to the Global Open-Source Security Community
• License: MIT License
• Copyright: Copyright (c) 2026 Aman Kumar Pandey