REAP

Recursive Extraction And Parsing — a general-purpose CLI tool for identifying and recursively extracting firmware and partition images. Works with raw eMMC/flash dumps, individual partition images, full-disk images (GPT or Rockchip PARM), and forensic disk images from a wide range of embedded Linux and Android devices.

Pure Python. No root, no FUSE, no mounting, no Linux kernel modules. Runs on macOS, Linux, and Windows.

What it does

Point it at a directory of partition .bin files, a single image, or a set of 7z archives and it will:

1. Identify each image's format through more than 60 ordered core detectors
2. Annotate what partition it is (boot, recovery, system, userdata, etc.) by reading ext-family superblock metadata and analyzing ramdisk contents
3. Extract contents recursively -- e.g. a boot image yields a kernel + ramdisk; the ramdisk decompresses to a cpio archive; the cpio extracts to a filesystem tree
4. Analyze kernels (version, config, build paths, kallsyms symbol table), bootloaders (U-Boot environment, embedded DTBs), and unknown partitions (forensic hex dump, strings, SHA256)
5. Report everything found in text, JSON, CSV, normalized findings/SARIF, opt-in CycloneDX inventory, and deterministic SHA-256 forensic manifests, including FBE encryption detection

Supported formats

Partition tables and disk layouts

Format	Detection	Extraction
GPT partition table	EFI PART at offset 0x200 or 0x1000 (UFS 4K sectors)	Individual partition images
MBR / DOS partition table	0x55AA at offset 510 + a valid primary entry (not a FAT VBR or GPT protective MBR)	Primary and bounded EBR logical partitions carved at their LBA-start offsets, with cycle/overlap checks, then recursed.
Apple Partition Map	Consistent ER Driver Descriptor Map and declared PM entry stride	Validates the complete map, driver coverage, logical/physical spans, overlap, and exact provider geometry before atomically carving data partitions
FreeBSD BSD disklabel	Redundant little-endian disklabel magic in sector 1 with an explicit 512--8192-byte sector size	Validates checksum, raw-provider geometry, relative offsets, and overlap before atomically carving nonempty partitions; embedded absolute-offset labels fail closed without parent-origin evidence
Rockchip PARM partition table	PARM at offset 0	Individual partition images (RK29xx/RK3xxx flash dumps)
Android super.img (LP metadata)	0x67446C70 at offset 0x1000	Logical partition images (system, vendor, product, etc.)
QCOW2 v2/v3 disk image	QFI\xFB at offset 0	Integrity-checked flattening of internal, uncompressed data and deterministic holes/zero clusters to a raw child image; unsupported backing, encryption, snapshots, external data, compression, and extended-L2 layouts fail closed
VirtualBox VDI v1.1 disk image	0xBEDA107F signature at offset 0x40	Flattens standard 1 MiB-block dynamic and static images to an exact raw child; unallocated/discarded dynamic blocks become zeros, while links/parents, saved/trailing regions, nonstandard extras, and ambiguous maps fail closed
VMware VMDK hosted sparse v1	KDMV at offset 0	Flattens a self-contained monolithicSparse extent to an exact raw child after authenticating its embedded descriptor, redundant directories/tables, physical ownership, and exact file envelope; split/flat, stream-optimized, compressed, snapshot/parent, external-extent, unclean, and trailing-data variants fail closed
Classic VHD fixed/dynamic disk image	Checksum-valid conectix footer at byte 0 or EOF; a tail-only dynamic match additionally requires an authenticated header	Flattens fixed and expandable type-2/type-3 images to an exact sparse raw child after footer, header, BAT, bitmap, bounds, and overlap validation; one damaged dynamic footer copy can be recovered unambiguously, saved-state sources are read without mutation, and differencing/parent/VHDX images fail closed
LVM2 physical volume	CRC-valid LABELONE / LVM2 001 label in sector 0--3	Extracts exact logical-volume children from a single PV with one data area, one or two validated metadata areas, and linear (striped, count 1) mappings; unsupported or ambiguous layouts fail closed before atomic publication

Android boot and kernel

Format	Detection	Extraction
Android Boot Image (v0--v4)	ANDROID! magic	Kernel, ramdisk, second-stage, recovery DTBO, DTB, and v4 GKI boot signature; legacy AOSP SHA-1 IDs are checked when present
Android Init Boot Image (v4)	ANDROID! v4 with a generic ramdisk and no kernel	Generic init ramdisk, distinguished from ordinary boot/recovery images
Android Vendor Boot Image (v3/v4)	VNDRBOOT magic	v3 vendor ramdisk + DTB; v4 bounded ramdisk-table fragments, DTB, and bootconfig with names/types/board IDs retained as metadata
Android update_engine full OTA	CrAU header (major versions 1/2 identified)	Major-version-2, minor-version-0 full payloads reconstruct exact target partition images from bounded REPLACE, bzip2, and XZ operations; delta/partial payloads fail closed without a validated source
ARM zImage	0x016F2818 at offset 0x24	Decompressed vmlinux, kernel config, version string, source paths, kallsyms, all strings
ARM64 Image	ARM\x64 at offset 0x38	Kernel config, version string, source paths, kallsyms, all strings
Raw ARM kernel binary	MSR CPSR instruction + Linux version string	Kernel config, version string, source paths, kallsyms, all strings
Device Tree Blob (DTB)	0xD00DFEED	Extracted DTB, optional dtc decompile to DTS
DTBO container	0xD7B7AB1E	Individual DT overlay entries

Bootloaders and firmware

Format	Detection	Extraction
U-Boot FIT/ITB	Structurally valid version-17 FDT with unique /images and /configurations nodes	Atomically publishes every inline or supported external image, verifies CRC32/SHA-1/SHA-2 hashes, inventories untrusted signature metadata, and validates configuration references; see FIT support
Microsoft UF2	Dual start magic in each 512-byte transport block	Reorders authenticated sequence blocks and publishes exact, address-preserving contiguous flash segments; unsupported flags, family conflicts, overlaps, padding, and malformed blocks fail closed
U-Boot uImage	0x27051956	Unwrapped payload (kernel, ramdisk, firmware, device tree, etc.)
U-Boot binary	U-Boot <version> string, 64 KB--4 MB	Default environment, embedded DTBs, strings
U-Boot environment	CRC32 + key=value pairs, power-of-2 size	Parsed environment variables
Samsung Exynos boot partition	BL1 header pointer + Exynos BL label	bl1.bin, u-boot.bin, tzsw.bin
Rockchip KRNL wrapper	KRNL at offset 0	Unwrapped payload (re-identified as gzip, zImage, etc.)
ELF binary	\x7fELF magic	Metadata dump (class, machine, entry point), strings
AVB vbmeta	AVB0 / AVBf	Metadata dump (version, algorithm, rollback index, flags)

Encrypted firmware containers

Format	Detection	Extraction
IM*H firmware container	IM*H at offset 0 or 0x400	Header parse (version, module name/type, chunk table, key family). Successful extraction preserves encrypted chunks (RTOS, kernel, TZOS, DTB, etc.) as raw .bin files; an explicitly selected and locked API-v2 dji_imah provider may additionally publish verified plaintext.
Ambarella environment (UNR0)	UNR0 + 0x5AA5 flags	Boot config, A/B slot status, firmware versions, bootloader logs
BitLocker (BDE) volume	-FVE-FS- OEM ID at offset 3, or BitLocker volume GUID at offset 0x10 of an NTFS / FAT-style boot sector	Boot-sector + FVE metadata parse: variant (Vista vs. Win 7+ / To-Go), volume GUID, encryption method (AES-128/256, CBC ± Elephant diffuser, XTS), dataset GUID, FVE block locations, full inventory of key protectors (TPM, TPM+PIN, password, recovery password, startup key, smart card, clear key, etc.). Surfaces clear-key suspended volumes prominently. Decryption is out of scope.
LUKS1 / LUKS2 volume	LUKS\xBA\xBE magic at offset 0; version word distinguishes LUKS1 from LUKS2	LUKS1: cipher / mode / hash / key size / UUID + per-keyslot PBKDF2 iterations, salt, active flag. LUKS2: binary header + JSON metadata — segments (cipher), keyslots (PBKDF2 vs. Argon2i / Argon2id with memory + parallelism), tokens (systemd-cryptenroll TPM2 / FIDO2 / PKCS11 bindings), digests. Decryption is out of scope.

Filesystems

Format	Detection	Extraction
ext2 / ext3 / ext4	0xEF53 at offset 0x438 plus authoritative feature masks	Full filesystem tree with checked legacy direct/indirect traversal and ext4 extents; journals and unsupported features fail closed; ext4 FBE is detected
FAT12/16/32	0xEB/0xE9 + 0x55AA at 510	Full filesystem tree (LFN support)
exFAT	EXFAT OEM ID at offset 3	Full filesystem tree (long names, fragmented + NoFatChain runs)
NTFS	NTFS OEM ID at offset 3 + 0x55AA at 510	Full filesystem tree (resident + non-resident $DATA, LZNT1 compression, $ATTRIBUTE_LIST, $INDEX_ROOT + $INDEX_ALLOCATION, ADS, USA fixup; EFS-encrypted files marked, reparse points recorded as text)
cramfs	0x28cd3d45 at offset 0 + Compressed ROMFS at 16	Full filesystem tree (zlib blocks, little- and big-endian)
romfs	-rom1fs- at offset 0	Full filesystem tree (files, dirs, symlinks, hard links)
squashfs	hsqs (little-endian) / sqsh (big-endian) at offset 0	Full filesystem tree for v4.0 little-endian images — inode/directory metadata + data/fragment blocks. Compressors: gzip, xz/lzma, lz4, zstd (if zstandard installed); lzo and big-endian images are flagged, not decoded
JFFS2	node magic 0x1985 (85 19 LE / 19 85 BE) at offset 0	CRC-validated filesystem recovery (common headers, inode/dirent nodes, names, and compressed data); newest valid versions win; none/zero/copy/zlib/rtime compressors are decoded, while lzo/rubin are flagged
ISO 9660 / Joliet / Rock Ridge	CD001 primary descriptor in the sector-16 descriptor set	Bounded single-volume tree recovery with Joliet UCS-2 and RRIP names, attributes, symlinks, and continuations; validated El Torito catalogs yield exact floppy images or explicitly bounded no-emulation loader spans
UFS1 / UFS2 (FFS)	Version magic at the canonical 8 KiB or 64 KiB primary-superblock location	Extracts clean standalone FreeBSD-compatible filesystems after validating cylinder groups, backup superblocks, allocation maps, inode reachability/link counts, direct/indirect blocks, paths, and source stability; journals, active recovery, ACL/xattr policy, and special nodes fail closed
XFS v5	XFSB at offset 0	CRC-validates every allocation-group superblock and inventories stable geometry, UUIDs, features, log placement, and counters. Namespace/file extraction is explicitly not yet supported and is reported as metadata-only.
UBI	UBI# erase-counter header at offset 0 (per PEB)	Reassembles each logical volume from its PEBs (infers PEB size, orders LEBs, parses volume names) and carves each volume as a child image for recursion
UBIFS	node magic 0x06101831 at offset 0	CRC-validated linear node-scan recovery (tolerates mid-commit/partial dumps where the committed index is inconsistent): rebuilds the tree from surviving newest-sqnum inode/dentry/data nodes, decompresses none/zlib (lzo/zstd if the optional lib is present), zero-fills + flags missing blocks, and puts unreferenced inodes under __ORPHANS__/
EROFS	0xE0F5E1E2 at offset 0x400	Flat plain/inline files plus conservative LZ4 full-index and 4-byte compact-index, one-pcluster mappings; advanced/chunk/fragment/2-byte layouts fail closed with typed warnings.
F2FS	0xF2F52010 at offset 0x400	CRC-validated checkpoint-pack recovery, checkpoint-authoritative NAT selection, direct/indirect/double-indirect nodes, sparse files, inline data and inline directories. Encrypted files, compressed clusters, and special inodes fail closed.
littlefs v2.0/v2.1	Canonical block-0 compacted superblock tags, exact declared logical geometry, and a commit-CRC-valid metadata-pair copy; damaged bootstrap routing requires a source-hash-bound geometry manifest	Read-only recovery of nested directories, inline files, CTZ skip-list files, revisions, splices, hard/soft tails, contiguous expanded-superblock prefixes, commit CRCs, and v2.1 forward CRCs; ambiguous geometry, interrupted global state, future semantics, overlap, and corruption fail closed before grouped atomic publication
SPIFFS	No geometry-free detector: a source-bound geometry manifest must authenticate the exact byte region and all 16 compile-time/layout parameters	Read-only regular-file recovery across object-index header/continuation pages and data pages, including slash-containing names, non-power-of-two geometry, deleted-page accounting, bounded validation, and grouped atomic no-clobber publication; geometry and unsupported object semantics are never guessed
YAFFS2	No geometry-free detector: a source-bound geometry manifest must authenticate the exact byte region and all 14 NAND/tag/ECC parameters	Conservative regular-file/directory recovery from OOB or in-band tags with sequence/revision selection and optional yaffs_ecc_other correction; checkpoint, link/special/shadow/shrink/bad-block, graph, path, and ambiguous-layout cases fail closed

Compression and archives

Format	Detection	Extraction
gzip	1F 8B	Decompressed content
LZ4 frame	04 22 4D 18	Decompressed content
LZ4 legacy	02 21 4C 18	Decompressed content (Android ramdisk format)
LZMA	5D 00 00	Decompressed content
bzip2	BZh	Decompressed content
XZ	FD 37 7A 58 5A 00	Decompressed content
cpio newc	070701 / 070702	Files, directories, symlinks (as text files with -> target)
7z archive	37 7A BC AF 27 1C	Full decompression (supports split .7z.001 parts)
ZIP / TAR / Odin .tar.md5	Native signatures and validated TAR trailer	Safely extracts regular members; rejects traversal, links/devices, collisions, corruption, excessive members and expansion; Odin MD5 is verified.
Zstandard	28 B5 2F FD	Bounded streamed decompression
Android sparse image	0xED26FF3A	Converted to raw image, then re-identified and extracted

Device-specific partitions

Format	Detection	Extraction
Android devinfo	ANDROID-BOOT! magic	Lock status, tamper flags
ArduPilot / Autel DataFlash flight log	0xA3 0x95 head + FMT type 0x80	Self-describing FMT schema, per-message CSVs, decoded firmware versions, and a GPS track (GPX + KML). Auto-detects the Autel 8-byte FMT name field vs. stock ArduPilot's 4-byte field.
Qualcomm modemst (EFS)	IMGEFS marker in first 64 bytes	Forensic scan (SHA256, strings, hex dump)
BMP image	BM + valid DIB header	Trimmed BMP (strips partition padding)
Boot logo container	ASCII count/sizes header + BMP at 0x200	Individual BMP images
Empty / zeroed	All-zero content	Verified-empty marker with likely purpose annotation

Installation

Requires Python 3.10+ (supported through Python 3.13).

From PyPI:

pip install reap-cli

The PyPI distribution is reap-cli because the bare reap name on PyPI is held by an unrelated, long-abandoned 2012 package. We are pursuing a PEP 541 transfer. The installed CLI command is reap regardless.

From source (for development):

git clone https://gitlab.com/blackbox-research/reap
cd reap
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Dependencies (installed automatically):

• ext2 / ext3 / ext4 -- pure-Python ext-family filesystem reader (no FUSE/mounting)
• lz4 -- LZ4 decompression for Android ramdisks
• zstandard -- Zstandard archives and filesystem blocks

Normal runs are core-only: REAP neither enumerates nor imports the reap.plugins entry-point group unless at least one repeatable --enable-plugin NAME is supplied. Inspect installed plugin metadata without importing plugin code using reap plugins; even there, --load executes code only when one or more names are explicitly selected with --enable-plugin. Extraction also accepts repeatable --disable-plugin NAME options.

Object-based plugin API v1 remains a trusted, serialized, in-process compatibility interface. Plugin API v2 is a closed subprocess protocol for the single decrypt-artifact capability: an explicitly selected provider must also match --plugin-lock, including its launcher size and SHA-256. Supplying a lock never enables a plugin by itself. The core hands ciphertext by descriptor, enforces finite control, time, staging, and output limits, verifies both sides' size/digest/identity claims, and atomically ingests plaintext. Provenance is core-owned and excludes key references and control streams. Process separation is not a same-UID security sandbox, and a launcher digest does not transitively pin an interpreter or its libraries; see docs/API.md and docs/LIMITATIONS.md.

The full frozen plugin inventory, including selection state, API/isolation/capability data, distribution and launcher identities, and lock digest, is bound into resume snapshots. Forensic manifests bind selected-plugin evidence and embed the canonical lock digest and document. Incompatible, conflicting, unknown, or unauthenticated selections fail closed with exit 7.

Usage

reap <input_path> [options]

input_path can be a single image file, a directory containing partition images, or a set of 7z archives.

Options

Flag	Description
-o DIR	Exact output directory for a single file; output root for a directory (defaults: <file>_unpacked/ or <directory>/unpacked/)
--identify-only	Print format identification only, no extraction
--skip-ext4	Skip ext4 filesystem extraction (useful for huge partitions)
--skip-archives	Skip 7z archive extraction
--force-archives	Force archive extraction even when physicalImage/ already exists
--no-recursive	Don't recurse into extracted children
--salvage, --carve-scan	For images with an unparseable partition table (identified as Unknown), scan for embedded ext-family/FAT/exFAT/NTFS filesystems and a UBI area (UBI# after a boot region) and carve them out for extraction
--max-depth N	Maximum recursion depth (default: 10)
--max-items N	Total identified-item admission budget; sorted top-level wave positions are reserved before workers start (default: 100000; 0 = unlimited)
--max-output-bytes N	Per-artifact extraction cap (default: max(1 GiB, 200× input); 0 = unlimited)
--max-total-output-bytes N	Thread-safe whole-run output cap (default: 20 GiB; 0 = unlimited)
--timeout SECONDS	Cooperative deadline; timeout exits 124 and cleans partial files
--progress none|human|json	Progress on stderr; JSON mode is JSON Lines
--resume	Resume only from a matching hash-validated journal; stale/missing state exits 3
--geometry-manifest PATH	Use a versioned, portable-path/size/SHA-256-bound geometry contract for raw littlefs, SPIFFS, or YAFFS2; unknown geometry is never guessed
--enable-plugin NAME	Explicitly opt into one installed plugin by entry-point name (repeatable); ordinary runs remain core-only
--disable-plugin NAME	Explicitly exclude one named plugin (repeatable)
--plugin-lock PATH	Authenticate explicitly enabled API-v2 subprocess providers; the lock never enables a provider by itself
-y, --assume-yes	Use the default archive-skip answer without prompting
-j, --jobs N	Parallel extraction workers (0=auto, 1=sequential; default: auto)
--fail-fast	Stop scheduling new deterministic worker waves after the first failed item tree; the failing wave is allowed to finish safely
-v	Verbose output (INFO level)
-vv	Debug output
--report text|json|csv|both|all	Report format (all emits all three; default: both)
--findings none|json|sarif|both	Publish normalized findings JSON and/or SARIF (default: json)
--fail-on-finding none|info|warning|error	After publishing evidence, exit 4 when a finding meets the selected severity threshold (default: none)
--sbom	Opt in to deterministic CycloneDX package and ELF inventory of completed output files
--sbom-max-files N	Maximum regular files inspected by --sbom (default: 100000)
--sbom-max-bytes N	Maximum aggregate bytes inspected by --sbom (default: 4 GiB)
--sbom-max-packages N	Maximum package-database records observed by --sbom before deduplication (default: 100000)
--sbom-max-elf-entries N	Maximum program-header, section-header, and dynamic-table entries inspected per ELF file (default: 8192)
--manifest none|json|csv|both	Portable SHA-256 evidence manifest (default: both)

The JSON report always embeds the normalized findings list and severity counts. Standalone findings.json is emitted by default; SARIF is available for scanner integrations. --findings none suppresses only standalone files, not report findings or policy evaluation. A finding policy is evaluated only after requested reports, findings, SBOM, and manifest evidence have been published, so exit 4 does not suppress the evidence that explains it; extraction-failure exit 2 takes precedence. --sbom is deliberately opt-in and all four inventory limits must be finite positive integers.

Resume diagnostics are separate read-only commands. They require explicit roots so portable journal paths cannot be resolved against ambient state:

reap resume status output/.reap-resume.json \
  --input evidence/ --output output/ --format json
reap resume explain output/.reap-resume.json '$INPUT/system.img' \
  --input evidence/ --output output/
reap resume prune output/.reap-resume.json \
  --input evidence/ --output output/              # dry-run
reap resume prune output/.reap-resume.json \
  --input evidence/ --output output/ --apply      # journal records only

status and explain never write. prune is a dry-run unless --apply is present, and even then it atomically removes only unusable journal records; it never deletes extracted files. Unusable records exit 4, while unsafe or malformed journals exit 5. JSON output carries stable reason_code values.

Directory output names are allocated before concurrency begins. REAP uses full Unicode compatibility caseless matching plus stable source-name hashes to avoid case, normalization, same-stem, and long-name collisions on portable evidence media. Existing output roots are rejected if any scanned component is a link or special file; REAP never follows or removes such entries.

Verify both recorded content and inventory completeness without modifying the case:

reap verify output/manifest.json --input evidence/ --output output/ --strict

New manifests fingerprint every regular file in the selected input scope and output tree, including non-recursed extracted artifacts, reports, and the resume journal. Each entry identifies its artifact class. --strict fails for any later unrecorded input or output; use the independent --added-inputs and --added-outputs policies when warning-only or ignore behavior is required. Geometry, diff, manifest, JSON-report, findings, CycloneDX SBOM, plugin-lock, plugin-result, and resume-journal Draft 2020-12 schemas are included in the installed package under reap/schemas; their stable IDs and versioning policy are documented in docs/SCHEMAS.md.

Compare two current manifest or JSON-report documents deterministically:

reap diff baseline/manifest.json candidate/manifest.json --format json
reap diff baseline/report.json candidate/report.json \
  --fail-on added --fail-on removed --fail-on findings

REAP auto-detects a common supported document type and validates both inputs before comparing them. Output can be human, json, or csv; --output publishes atomically and refuses an existing path unless --replace is given. Differences alone exit 0. A selected --fail-on category exits 4, malformed, unsupported, ambiguous, or unsafe input/output exits 5, and invalid CLI usage exits 2. Policy categories are added, removed, changed, moved, partition, path, format, metadata, finding_added, finding_removed, finding_changed, findings, and any.

Detached authenticity is opt-in and never creates or discovers keys:

pip install 'reap-cli[signing]'
chmod 600 investigator-ed25519-private.pem
reap sign-manifest output/manifest.json --private-key investigator-ed25519-private.pem --signature output/manifest.sig.json
reap verify-signature output/manifest.json --signature output/manifest.sig.json --public-key trusted-investigator-ed25519.pem

Private keys must be unencrypted Ed25519 PEM files with owner-only permissions on POSIX systems. Keep them outside evidence/output trees. Public trust is always an explicit --public-key path; key generation, persistence, lookup, and logging are deliberately out of scope. Signature mismatch exits 6; malformed inputs, unsafe key permissions, missing signing support, and I/O errors exit 5. Existing reap verify content verification and exit codes are unchanged and can be run before or after authenticity verification.

Examples

Identify all partitions in a dump:

reap ./physicalImage --identify-only

Full extraction (skip large ext4 partitions):

reap ./physicalImage --skip-ext4 -v

Extract a single boot image:

reap boot.img -o ./boot_extracted -v

Extract a directory of 7z archives (split parts supported):

reap ./archives/ -v

Parallel extraction with 4 workers:

reap ./physicalImage -j 4 -v

Output structure

For a boot image, the recursive extraction produces:

boot_unpacked/
    kernel_info.txt          # Kernel analysis summary
    kernel_config.txt        # Build-time .config (if IKCONFIG enabled)
    kernel_source_paths.txt  # Build-time source paths
    kernel_strings.txt       # All embedded ASCII strings
    kallsyms.txt             # Kernel symbol table (if present)
    vmlinux                  # Decompressed kernel binary
    ramdisk_unpacked/
        init                 # cpio filesystem tree
        init.rc
        fstab.*
        sbin/
        ...

For a directory of partitions, you get a subdirectory per partition plus reports:

physicalImage/unpacked/
    report.txt               # Human-readable report
    report.json              # Machine-readable report
    report.csv               # Flat machine-readable report (with --report csv/all)
    findings.json            # Normalized findings (default)
    findings.sarif.json      # SARIF findings (with --findings sarif/both)
    sbom.cdx.json            # CycloneDX package/ELF inventory (with --sbom)
    manifest.json            # Versioned portable evidence manifest
    manifest.csv             # Flat evidence manifest
    .reap-resume.json        # Private bounded resume journal
    mmcblk0p1_unpacked/      # boot image contents
    mmcblk0p2_unpacked/      # DTB contents
    mmcblk0p3_unpacked/      # recovery image contents
    mmcblk0p4_unpacked/      # system filesystem tree
    ...

Partition annotation

The tool automatically identifies partition roles by:

• Reading the ext2/3/4 superblock s_last_mounted field (e.g. /system, /data, /cache)
• Analyzing boot image ramdisks for /sbin/recovery to distinguish boot vs recovery
• Parsing U-Boot uImage type fields (kernel, ramdisk, firmware, device tree)
• Parsing IM*H firmware module names and types (bootloader, kernel, RTOS)
• Recognizing format-specific roles (DTB, vbmeta, DTBO, sparse, super, modemst)
• Inferring empty partition purpose from size (<=4 MB zeroed = likely misc or metadata)

Annotations appear in reports and verbose output as labels like (recovery), (system), (userdata), etc.

FBE encryption detection

When extracting ext4 filesystems with File-Based Encryption (FBE), the tool:

• Detects the encryption superblock flag and per-inode encryption flags
• Hex-encodes encrypted filenames for safe extraction
• Writes encrypted_paths.txt listing all encrypted files and directories
• Reports encryption algorithms (AES-256-XTS, AES-256-GCM, etc.) in JSON output

Architecture

reap/
    cli.py              # Argument parsing, entry point
    identify.py         # Ordered detection registry (65 core formats)
    annotate.py         # Partition role inference
    pipeline.py         # Recursive extraction orchestrator (parallel workers)
    report.py           # Atomic text + JSON + CSV reports (FBE-aware)
    findings.py         # Normalized JSON/SARIF findings and policy evaluation
    inventory.py        # Bounded CycloneDX package/ELF inventory
    manifest.py         # Deterministic portable JSON + CSV evidence manifests
    diff.py             # Validated deterministic manifest/report comparisons
    limits.py           # Output budgets, bounded writers, cancellation
    resume.py           # Bounded flat v2 resume journals and diagnostics
    plugin_lock.py      # Closed API-v2 provider/launcher lock contract
    plugin_protocol.py  # Descriptor handoff and verified atomic ingest
    handlers/
        __init__.py     # BaseHandler ABC, handler registry
        ambarella_env.py # Ambarella UNR0 boot environment
        avb.py          # AVB vbmeta metadata
        bitlocker.py    # BitLocker (BDE) detection + FVE metadata + protectors
        bmp.py          # BMP image (partition padding trim)
        boot_img.py     # Android boot image (v0--v4)
        bootlogo.py     # Boot logo container (multiple BMPs)
        compression.py  # gzip, LZ4, LZMA, bzip2, XZ
        cpio_handler.py # cpio newc archives
        dataflash.py    # ArduPilot/Autel DataFlash flight log (schema, CSVs, GPS track)
        devinfo.py      # Android devinfo (lock status)
        dji_imah.py     # IM*H ciphertext preservation + optional locked provider
        dtb.py          # Device Tree Blob
        dtbo.py         # DTBO container
        elf.py          # ELF binary metadata + strings
        ext4_handler.py # ext2/3/4 (legacy blocks, extents, FBE detection)
        exynos_boot.py  # Samsung Exynos eMMC boot partition
        fit.py          # Validated U-Boot FIT/ITB images and configurations
        apm.py          # Apple Partition Map carving
        bsd_disklabel.py # FreeBSD BSD disklabel carving
        mbr.py          # MBR/DOS partition table (splits primary partitions)
        fat.py          # FAT12/16/32 filesystem
        gpt.py          # GPT partition table (512-byte + 4K UFS sectors)
        luks.py         # LUKS1 + LUKS2 detection + keyslot / token metadata
        lvm2.py         # Single-PV linear LVM2 logical volumes
        littlefs.py     # Conservative littlefs v2.0/v2.1 read-only recovery
        spiffs.py       # Explicit-geometry, read-only SPIFFS recovery
        yaffs2.py       # Explicit-geometry, read-only YAFFS2 recovery
        ota_payload.py  # Android update_engine full OTA target partitions
        qcow2.py        # Integrity-checked QCOW2 v2/v3 flattening
        vdi.py          # Standard dynamic/static VDI v1.1 flattening
        vmdk.py         # Self-contained hosted sparse VMDK v1 flattening
        vhd.py          # Fixed/dynamic classic VHD flattening
        uf2.py          # Address-preserving Microsoft UF2 extraction
        ufs.py          # Clean standalone UFS1/UFS2 recovery
        xfs.py          # XFS v5 redundant-superblock metadata validation
        modemst.py      # Qualcomm modem EFS partition
        ntfs.py         # NTFS filesystem (MFT + runlists + LZNT1 + indexes)
        raw.py          # Empty + unknown fallback (forensic scan)
        salvage.py      # --salvage carve-by-scan for unparseable partition tables
        raw_kernel.py   # Raw ARM kernel binary
        rk_krnl.py      # Rockchip KRNL wrapper
        rkparm.py       # Rockchip PARM partition table
        seven_zip.py    # 7z archive (split-part support)
        sparse_img.py   # Android sparse -> raw conversion
        super_img.py    # super.img LP metadata
        uboot_bin.py    # U-Boot binary (environment, embedded DTBs)
        uboot_env.py    # U-Boot environment block
        uimage.py       # U-Boot uImage wrapper
        zimage.py       # ARM zImage / ARM64 Image kernel extraction
        _kernel_utils.py # Shared kernel analysis (version, config, kallsyms)

Each handler implements BaseHandler.extract() and returns an ExtractionResult with optional children for recursive processing. Versioned public interfaces, plugin keys and conflict rules are documented in docs/API.md.

The pipeline orchestrator (pipeline.py) drives identify → annotate → dispatch → recurse. Children can run concurrently while sharing one reservation-based output ledger and cancellation token. See architecture, schemas, and limitations.

Symlink handling

Symlinks found inside ext-family filesystems and cpio archives are not created as OS symlinks (which can cause issues on some platforms and create security risks with path traversal). Instead, they're written as small text files containing -> target and recorded in the extraction metadata / JSON report.

Running tests

pip install -e ".[dev]"
pytest tests/ -q

The compact, risk-based suite covers format contracts, hostile-input bounds, pipeline orchestration, and forensic scanning. Provenance-pinned real-corpus fixtures provide authentic-tool differentials without requiring external tools during an ordinary test run. A lean-policy checkpoint consolidated expansion- heavy suites without dropping their behavioral tables. CI folds coverage and generated-corpus validation into the Python 3.13 compatibility job instead of repeating a fifth full suite. The release harness runs one full/static preflight, the minimum two-build reproducibility comparison, and then only the 20 concurrent installed-package workflows. Protected tag pipelines reuse the exact commit's successful main validation instead of rerunning the compatibility suite and quality build. See the testing policy.

Known limitations

• EROFS, F2FS, littlefs, SPIFFS, YAFFS2, UFS, VDI, and VMDK: Useful conservative subsets are extracted. SPIFFS and YAFFS2 require authenticated complete geometry manifests; advanced or ambiguous layouts intentionally fail closed. XFS currently validates and inventories v5 superblock metadata but does not extract its namespace. See the exact support matrix.
• Encrypted partitions: FBE-encrypted ext4 partitions are detected and documented, but file contents remain encrypted. The tool does not perform Android FDE/FBE decryption. EFS-encrypted files on NTFS volumes are similarly skipped, with a per-file marker recording the encryption. BitLocker and LUKS volumes are detected and their full encryption-scheme metadata is surfaced, but the encrypted payload itself is not decrypted — pipe the source image through dislocker / libbde (BitLocker) or cryptsetup (LUKS) for that.
• IM*H decryption: On a successful extraction, core parsing preserves encrypted chunks. Optional plaintext requires an explicitly selected, lock-authenticated API-v2 dji_imah provider and operator-supplied key reference; REAP distributes no AES keys. A provider protocol failure fails the handler transaction closed and publishes neither raw nor plaintext outputs.
• Large partitions: Extracting a 54 GB ext4 partition takes time and disk space. Use --skip-ext4 to skip these, or extract individual partitions as needed.
• HFS+ and other deferred filesystems: Native HFS+ extraction is not claimed. Journal state, HFS wrappers, catalog/extents B-trees, and Unicode/case semantics make a rushed core parser disproportionate to the recovery evidence goal; a future route should be an explicit, audited read-only adapter. APFS, Btrfs, and multi-device filesystem assembly remain deferred.
• External tools: Split 7z requires system 7z; single files can fall back to py7zr, and DTB decompilation can invoke dtc. REAP bounds these subprocesses and validates staged output, but does not digest-pin the 7z or dtc executable.
• Symlinks: Recorded as text files, not created as actual OS symlinks.
• Text files: Plain-text metadata files (.txt, .sha256, .xml, README) in the input directory are detected and skipped rather than subjected to forensic extraction.