Skip to main content

filegrail

Local file provenance and metadata analysis.

PyPI Python 3.10+ 68 formats Runtime dependencies Local and read-only Network requests CI License

What it does · Evidence sources · Formats · Install · Usage · Examples


filegrail analyzes files and directories using two kinds of information:

  1. Traces already stored on the machine - browser downloads, OS origin metadata, shell history, archives, torrents, recent files, sync folders and trash records.
  2. Data stored inside the files - EXIF, XMP, IPTC, C2PA, document properties, media metadata, email headers and other embedded metadata.

It keeps three questions separate:

Area Question
Acquisition How did the file reach this machine?
Intrinsic What does the file reveal about its own earlier history?
Interaction What happened to the file after it arrived?

Scanning is local, read-only and makes no network requests. filegrail clean is the only command that writes files, and it writes cleaned copies to a separate directory.


What it does

Capability What you get
File provenance Download URLs, referrers, browser records, OS origin attributes, archive/torrent membership, shell activity
Metadata extraction Camera/device data, GPS, timestamps, authors, editors, software, document properties, media tags, C2PA and more
Evidence correlation Agreement and conflicts between independent sources
Timeline Acquisition, creation, editing and interaction events in chronological order
Identifiers URLs, domains, email addresses, IPv4 addresses, coordinates and hashes
File relationships XMP document IDs and derivation chains between related files
Shared-source clustering Files grouped by camera body, camera model or author
Comparison Metadata, provenance and timing differences between two files
Explanation Every source behind a finding for one file
Content search Identifiers extracted from supported document content with their location inside the file
Metadata removal Cleaned copies of supported files, with verification of what remains
JSON output Machine-readable results for scans and all commands
Evidence coverage doctor shows which local evidence sources are available and how far back they reach

Evidence sources

Acquisition and interaction traces

Source What filegrail can read
Browser download history Download URL, referrer, time and recorded size from Chrome, Chromium, Brave, Edge, Vivaldi and Firefox profiles
Windows Zone.Identifier HostUrl, ReferrerUrl, ZoneId
macOS Where From Download URL and referrer stored in kMDItemWhereFroms
macOS quarantine Download URL, referrer, downloading application and quarantine time
Linux XDG attributes user.xdg.origin.url and user.xdg.referrer.url
Archives Archive membership matched by file name and uncompressed size; archive origin can be inherited by extracted files
Torrent files and client stores Torrent membership, trackers, client, comments, info hash/magnet data; local qBittorrent, Transmission and Deluge stores
yt-dlp sidecars Page URL, uploader/channel, publication date, extractor and fetch time from .info.json
Shell history Fetch commands such as curl, wget, yt-dlp, scp, rsync, git, gh, aws and others; non-fetch commands are recorded as later interaction
Recent documents Linux desktop recent-file records
Windows Recent shortcuts .lnk records showing that a file was opened
Sync folders Nextcloud, Dropbox, Syncthing and OneDrive folder/account context
Trash records Previous path and deletion time from the freedesktop trash
Messenger file names WhatsApp and Telegram Desktop filename patterns; treated as weak evidence only

Use:

filegrail doctor

to see which of these sources are actually available on the current machine or profile.

For another user profile or a mounted copy:

filegrail doctor --home /mnt/profile
filegrail /mnt/evidence --home /mnt/profile

Supported formats

Embedded metadata

Metadata block Extensions Data extracted
EXIF .jpg .jpeg .jpe .tif .tiff .dng .nef .cr2 .arw .orf .rw2 .webp .heic .heif .avif Camera make/model, body serial, lens, software, capture time, GPS
PNG text .png .apng tEXt, zTXt, iTXt: software, creation time, author and other stored values
ISO BMFF .mp4 .m4v .mov .qt .3gp .m4a .heic .heif .avif Encoder/device, creation time, ISO 6709 location
Matroska .mkv .mk3d .webm .mka Writing application/library, segment date, tags
RIFF/BWF .wav .wave .rmi .avi LIST/INFO, recorder information, coding history, embedded ID3 where present
Vorbis comments .flac .ogg .oga .opus .spx Vendor string and all NAME=value comments
ID3 .mp3 .aac .tta Encoding software, artist, title, date and other ID3v2 frames
PDF Info .pdf PDF Info dictionary
OOXML properties .docx .docm .dotx .xlsx .xlsm .xltx .pptx .pptm Application, author, last editor, company, template, revision count, editing time
OLE properties .doc .dot .xls .xlt .ppt .pot .pps .msg Summary and document-summary properties
OpenDocument metadata .odt .ods .odp .odg .odf .ott .otp Generator, author, creation and editing metadata
EPUB package .epub OPF package metadata
RTF metadata .rtf Generator and \info fields
SVG metadata .svg Generator and embedded RDF metadata
Jupyter notebook .ipynb Kernel name and language runtime version
C2PA .jpg .jpeg .png Producing application, creation data, digital source type and hard-binding check

Cross-format metadata

These blocks are detected wherever they are embedded, not only by extension.

Block Typical data
XMP Creating application, author, title, document IDs and derivation information
XMP history Recorded editing steps and timestamps
IPTC By-line, credit, source, copyright, headline, caption, keywords, place and creation date

Email

Extension Data extracted
.eml Every Received: hop, connecting addresses and message headers
.msg Transport headers where present, plus OLE document properties

Archives

Extensions What is read
.zip .jar .whl Member names, sizes and metadata from supported files inside the archive
.tar .tgz .gz .bz2 .xz Members and supported metadata through the archive/compression layer

Files inside supported archives are analyzed without unpacking the archive to disk.

Torrents and sidecars

File Data extracted
.torrent Trackers, creating client, comment, torrent membership and magnet/info-hash data
<name>.info.json yt-dlp page URL, uploader/channel, publication date, extractor and fetch time

For the complete format reference and edge cases, see docs/FORMATS.md.


Content extraction

--content reads supported document text in addition to metadata and runs identifier extraction over it.

filegrail ./case --content

--content implies --identify.

Extensions Content read
.txt .text .md .markdown .rst .log Text by line
.csv .tsv .json .ndjson .jsonl .ipynb .yaml .yml .toml .ini .cfg .conf .vcf .ics Text/data by line
.html .htm .xhtml .xml .svg Visible text and relevant URLs/attributes
.docx .docm .dotx Body, footnotes, endnotes and comments
.xlsx .xlsm .xltx Shared strings and inline cell text
.pptx .pptm Slide text and notes
.odt .ods .odp .odg .odf .ott .otp Document body, headers and footers
.epub Chapters
.eml .msg Decoded message body

PDF metadata is supported, but PDF body text is not extracted by --content.


Analysis

Correlation and conflicts

filegrail compares independent evidence instead of collapsing everything into one origin field.

It can report:

  • multiple sources supporting the same origin
  • conflicting origin URLs
  • file-size mismatches
  • filename-only matches
  • creation/modification dates in impossible order
  • XMP editing steps out of sequence
  • EXIF vs XMP differences
  • PDF Info vs XMP differences
  • XMP derivation relationships between files
  • C2PA hard-binding mismatches

Confidence values rank competing claims. They are not probability scores or forensic verdicts.

Investigation pivots

filegrail ./case --identify

Extracts:

  • URLs
  • domains
  • email addresses
  • IPv4 addresses
  • coordinates
  • MD5, SHA-1 and SHA-256 values found in metadata/content

Each value keeps the file, source and field/location it came from.

Shared sources

filegrail ./photos --cluster

Groups files by:

  • camera body/serial
  • camera model
  • author

Timeline

filegrail ./case --timeline

Places recorded acquisition, creation, editing and interaction events in chronological order.

File relationships

XMP identifiers such as:

  • xmpMM:DocumentID
  • xmpMM:InstanceID
  • xmpMM:OriginalDocumentID
  • xmpMM:DerivedFrom

can link renamed or exported files and show relationships such as derived-from, source-of, same-document or common-ancestor.


Installation

Requires Python 3.10+.

pipx install filegrail

or:

uv tool install filegrail

From the repository:

git clone https://github.com/osint-shifu/filegrail.git
cd filegrail
PYTHONPATH=src python -m filegrail.cli /path/to/files

Runtime dependencies: 0.


Usage

filegrail <path> [options]
filegrail <command> [options]

Analyze one file:

filegrail suspicious.pdf

Analyze a directory recursively:

filegrail ./evidence

Analyze the current directory:

filegrail .

Running filegrail with no arguments displays the command overview and does not start a scan.

Commands

Command Purpose
filegrail PATH Scan one file or directory
filegrail scan PATH Explicit scan command
filegrail explain FILE Show the evidence behind findings for one file
filegrail compare A B Compare two files
filegrail doctor Show available local evidence sources
filegrail clean PATH --out DIR Write metadata-cleaned copies
filegrail clean PATH --check Check what cleaning would remove without writing files
filegrail menu Interactive command menu
filegrail help COMMAND Command-specific help

Scan options

Option Purpose
-v, --verbose Show every evidence record
--brief Index only, without per-file detail
--timeline Chronological event view
--identify Extract investigation identifiers
--content Also inspect supported document content; implies --identify
--cluster Group files by shared cameras/authors
--unknown-only Show only files with no findings
--hash Compute SHA-256 for each file
-j, --json JSON output
--redact Redact credentials before printing
--type NAME Filter by archive, audio, document, image, mail, text or video
--ext LIST Filter by extensions, e.g. --ext jpg,pdf
--limit N Limit files with no findings; 0 means all
--home DIR Read evidence from another user profile
--no-recurse Do not scan subdirectories
--no-skip Include normally skipped build/cache/vendor directories
--no-shell-history Disable shell-history correlation
--no-archives Disable archive-origin inheritance
--color Force ANSI color
--no-color Disable ANSI color

Clean options

Option Purpose
--out DIR Output directory for cleaned copies
--check Check cleaning without writing files
--overwrite Replace an existing destination file
--type NAME Filter by file family
--ext LIST Filter by extension
--no-recurse Do not descend into subdirectories
-j, --json JSON output

Examples

# Full analysis of one file
filegrail photo.jpg

# Scan a case and show only the index
filegrail ./case --brief

# Extract investigation pivots
filegrail ./case --identify

# Also search supported document content
filegrail ./case --content

# Build a timeline
filegrail ./case --timeline

# Find files sharing cameras or authors
filegrail ./case --cluster

# Explain exactly why a finding was produced
filegrail explain document.pdf

# Compare two files
filegrail compare original.docx edited.docx

# Analyze another user profile
filegrail /mnt/evidence --home /mnt/profile

# JSON report
filegrail ./case --json > report.json

# JSON report with credentials redacted
filegrail ./case --redact --json > report.json

Metadata removal

filegrail clean removes supported metadata from copies. Originals are never modified.

Cleanable formats

Family Extensions
JPEG .jpg .jpeg .jpe
PNG .png .apng
ISO BMFF media .mp4 .m4v .m4a .mov .qt .3gp
Microsoft OOXML .docx .docm .dotx .xlsx .xlsm .xltx .pptx .pptm
OpenDocument .odt .ods .odp .odg .ott .otp

Clean one file:

filegrail clean photo.jpg --out ./clean

Check without writing:

filegrail clean ./publish --check

After cleaning, the output is read again by the same metadata readers. Anything still detected is reported.

Metadata removal is not anonymization. Image pixels, sensor patterns, codec fingerprints and document content can still identify a source.


JSON and automation

--json is available on all main commands.

Schemas include:

  • filegrail.scan/1
  • filegrail.explain/1
  • filegrail.compare/1
  • filegrail.doctor/1
  • filegrail.clean/1

JSON output preserves:

  • files and paths
  • acquisition/intrinsic/interaction claims
  • decoded metadata
  • evidence sources
  • confidence values
  • findings and conflicts
  • extracted identifiers
  • shared-source clusters
  • file relationships

Privacy

filegrail makes no network requests.

Reports can contain sensitive local data such as:

  • private URLs
  • credentials or tokens embedded in URLs/commands
  • file-system paths
  • email addresses
  • IP addresses
  • GPS coordinates

Use:

filegrail ./case --redact

or:

filegrail ./case --redact --json

before sharing output.

Always review redacted reports manually before publishing them.


Limits

  • filegrail can only use evidence that still exists.
  • Cleared browser history, removed extended attributes or missing shell history cannot be reconstructed.
  • A supported sync folder can show account/folder context, but not who uploaded a file.
  • WhatsApp/Telegram filename patterns do not identify a sender or conversation.
  • C2PA hard binding is checked, but the certificate chain/signature trust is not verified.
  • PDF metadata is read; PDF body text is not extracted by --content.
  • Unsupported file formats can still participate in provenance analysis when external/local evidence about them exists.
  • filegrail is not a monitoring agent, chain-of-custody system or full disk-forensics suite.

Documentation


License

Apache-2.0.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filegrail-0.6.0.tar.gz (381.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

filegrail-0.6.0-py3-none-any.whl (214.4 kB view details)

Uploaded Python 3

File details

Details for the file filegrail-0.6.0.tar.gz.

File metadata

  • Download URL: filegrail-0.6.0.tar.gz
  • Upload date:
  • Size: 381.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for filegrail-0.6.0.tar.gz
Algorithm Hash digest
SHA256 be6f297c0ccef30d34223096e804ff69942fdfbb4e172021e1d7ae0dcd4aba6d
MD5 815bf642e92b2301a84c789dcb512fc8
BLAKE2b-256 0a057f3c64048c73fce2ba6146ab23d4fbb285881133da9404d48752aef2e79b

See more details on using hashes here.

Provenance

The following attestation bundles were made for filegrail-0.6.0.tar.gz:

Publisher: release.yml on osint-shifu/filegrail

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file filegrail-0.6.0-py3-none-any.whl.

File metadata

  • Download URL: filegrail-0.6.0-py3-none-any.whl
  • Upload date:
  • Size: 214.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for filegrail-0.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e33d1e3684c04e5abf0255a7ecaa37279c17857aa0182cc5d07abab53ee398cd
MD5 92fbc65b6da6507e18eb471e891e1719
BLAKE2b-256 4eb580edb993c2e23366f56ed340f5c8aa2aa90cf3fe6928e18c0fb5e671902c

See more details on using hashes here.

Provenance

The following attestation bundles were made for filegrail-0.6.0-py3-none-any.whl:

Publisher: release.yml on osint-shifu/filegrail

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

0.6.1

2 files

This release

0.6.0 This release

2 files

0.5.2

2 files

0.5.1

2 files

0.5.0

2 files

0.4.1

2 files

0.4.0

2 files

0.3.0

2 files

0.2.0

2 files

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page