FileSense
A fast, local CLI-based file analysis and deduplication engine.
Scan any directory, instantly find duplicate files, visualize storage usage, and reclaim wasted disk space — all from your terminal.
Table of Contents
- What is FileSense?
- Installation
- Quick Start
- CLI Commands
- Project Structure
- For Developers
- How It Works
What is FileSense?
FileSense helps you understand and clean up your storage:
- Finds duplicate files using a multi-stage hashing pipeline (size → partial hash → full SHA-256), much faster than naive full-file hashing.
- Breaks down storage usage by file type, category, and directory.
- Identifies your largest files instantly.
- Maintains a persistent index (SQLite) so repeated scans are near-instant — unchanged files are never rehashed.
- Never deletes anything automatically — a dry-run preview is always shown first.
Installation
Option A — Python Users (Recommended)
Requires Python 3.12 or higher.
pip install filesense-cli
That's it. The filesense command is now available from any directory in your terminal.
Verify the installation:
filesense --help
Option B — Direct Binary (No Python Required)
If you do not have Python installed, download the pre-built standalone executable for your operating system.
Step 1 — Go to the Releases page:
👉 https://github.com/Jkotwal01/FileSense/releases/latest
Step 2 — Download the correct file for your OS:
| Operating System | File to Download |
|---|---|
| Windows | filesense-windows-amd64.exe |
| macOS | filesense-macos-amd64 |
| Linux | filesense-linux-amd64 |
Step 3 — Run it:
Windows:
# Open PowerShell in your Downloads folder and run:
.\filesense-windows-amd64.exe --help
# Or scan a directory directly:
.\filesense-windows-amd64.exe scan "C:\Users\YourName\Downloads"
macOS / Linux:
# First, make the file executable (one-time setup):
chmod +x filesense-macos-amd64
# Then run it:
./filesense-macos-amd64 --help
# Or scan a directory:
./filesense-macos-amd64 scan ~/Downloads
Tip: To use
filesensefrom anywhere without typing the full file name, move the binary to a directory on your system PATH:
- Windows: Move the
.exetoC:\Windows\System32\or add its folder to your PATH environment variable.- macOS/Linux: Move it to
/usr/local/bin/filesense
Quick Start
The most common use case — scan a directory and find duplicates:
# 1. Scan your Downloads folder
filesense scan "C:\Users\Name\Downloads"
# 2. See which files are duplicates
filesense duplicates "C:\Users\Name\Downloads"
# 3. See a full storage breakdown
filesense analyze "C:\Users\Name\Downloads"
# 4. Preview what could be deleted (safe — no files are touched)
filesense clean "C:\Users\Name\Downloads" --dry-run
CLI Commands
Understanding the Syntax
filesense <command> <path> [options]
<path>— Required. The directory to operate on. Use.for the current directory.[options]— Optional flags. Do not type the brackets.
1. scan — Discover and Index Files
Recursively scans a directory, collects file metadata, detects duplicates, and saves everything to a local SQLite cache for fast future scans.
filesense scan <path> [--workers N] [--follow-symlinks] [--verbose]
| Option | Default | Description |
|---|---|---|
--workers N / -w N |
4 |
Number of parallel hashing threads. Use 8 for fast SSDs. |
--follow-symlinks |
off | Follow symbolic links during traversal. |
--verbose / -v |
off | Print detailed debug logs for each file processed. |
Examples:
# Basic scan of current directory
filesense scan .
# Scan Downloads with 8 threads (faster on SSDs)
filesense scan "C:\Users\Name\Downloads" --workers 8
# Verbose scan to see every file being processed
filesense scan ./projects --verbose
2. duplicates — Find Duplicate Files
Displays all duplicate file groups, showing which files are identical, their sizes, and how much space you can recover by removing them.
filesense duplicates <path> [--workers N]
Example:
filesense duplicates "C:\Users\Name\Documents"
Sample output:
Group #1 a3f9c2d1... 150.2 MB x 3 copies -> 300.4 MB recoverable
KEEP C:\Users\Name\Documents\project\report.pdf
DUP C:\Users\Name\Desktop\report_copy.pdf
DUP C:\Users\Name\Downloads\report (1).pdf
3. analyze — Storage Breakdown
Shows a breakdown of storage usage grouped by file category (Images, Videos, Documents, etc.) and file extension (.pdf, .mp4, .zip, etc.).
filesense analyze <path>
Example:
filesense analyze "C:\Users\Name\Downloads"
4. largest — Top Space Consumers
Lists the N largest individual files in a directory, sorted by size.
filesense largest <path> [--top N]
| Option | Default | Description |
|---|---|---|
--top N / -n N |
10 |
Number of files to show. |
Examples:
# Show top 10 largest files
filesense largest .
# Show top 25 largest files
filesense largest "C:\Users\Name" --top 25
5. report — Full Report
Runs a complete analysis in one command: scan, duplicate detection, storage breakdown, and largest files.
filesense report <path> [--workers N]
Example:
filesense report "C:\Users\Name\Documents" --workers 8
6. benchmark — Performance Comparison
Measures the speed difference between a naive approach (hashing every file fully) and FileSense's multi-stage pipeline. Useful for understanding the performance gains on your specific hardware.
filesense benchmark <path> [--workers 1,2,4,8]
| Option | Default | Description |
|---|---|---|
--workers <list> |
1,2,4 |
Comma-separated list of thread configs to test. |
Examples:
filesense benchmark ./data
filesense benchmark ./data --workers 1,4,8,16
7. clean — Remove Duplicates Safely
Previews or performs cleanup of duplicate files. Default is always dry-run — no files are ever touched unless you explicitly confirm.
# SAFE — Preview only (default). Shows what WOULD be deleted.
filesense clean <path> --dry-run
# DANGEROUS — Actual deletion. Requires typing 'y' at a confirmation prompt.
filesense clean <path> --no-dry-run
⚠️ Safety guarantee: FileSense always keeps the file with the shortest path (the most likely original) and only deletes confirmed duplicates. Even with
--no-dry-run, you must manually typeyto confirm before any file is deleted.
8. index status — Database Info
Shows statistics about the local SQLite cache stored at ~/.filesense/filesense.db.
filesense index status
Sample output:
Database C:\Users\Name\.filesense\filesense.db
Indexed files 12,482
Hashed files 9,341
Total size 47.2 GB
Project Structure
FileSense/
├── filesense/
│ ├── metadata.py # FileMetadata dataclass + file type classifier
│ ├── scanner.py # FileScanner — recursive directory traversal
│ ├── hasher.py # HashManager, PartialHashStrategy, FullHashStrategy
│ ├── index.py # PersistentIndex — SQLite cache (~/.filesense/)
│ ├── duplicate.py # DuplicateDetector — 3-stage pipeline
│ ├── storage.py # StorageAnalyzer — usage metrics
│ ├── report.py # ReportGenerator — Rich terminal output
│ ├── benchmark.py # BenchmarkEngine — performance measurement
│ ├── logging_config.py # Rotating log handler (logs/filesense.log)
│ └── cli.py # Typer CLI — all 8 commands
├── tests/ # 55 unit tests
├── docs/
│ └── INTERVIEW_GUIDE.md # In-depth technical deep dive
├── main.py # Entry point
├── pyproject.toml # Package configuration
└── requirements.txt # Dependencies
For Developers
Setup
git clone https://github.com/Jkotwal01/FileSense.git
cd FileSense
pip install -r requirements.txt
Running from source (without installing)
# Windows
python -X utf8 main.py scan .
# macOS / Linux
python main.py scan .
Running Tests
# Run all 55 tests
python -m pytest tests/ -v
# Run a specific test file
python -m pytest tests/test_scanner.py -v
python -m pytest tests/test_hasher.py -v
python -m pytest tests/test_duplicate.py -v
python -m pytest tests/test_storage.py -v
How It Works
FileSense uses a 3-stage filtering pipeline that minimizes disk I/O:
All files
│
▼
Stage 1: Group by file SIZE ← Free (no disk reads)
│ Unique-size files eliminated immediately
│
▼
Stage 2: Partial Hash (64 KB) ← Reads only 32 KB from head + 32 KB from tail
│ Most non-duplicates eliminated cheaply
│
▼
Stage 3: Full SHA-256 ← Only reads files that survived both filters
│ Confirms exact duplicates
│
▼
Duplicate groups (sorted by recoverable space)
On a 50 GB dataset, a naive approach reads 50 GB from disk. FileSense typically reads only ~5 GB — a 90%+ reduction in disk I/O.
For a complete technical breakdown of every class, algorithm, and design decision, see docs/INTERVIEW_GUIDE.md.
Metadata
Release files for filesense-cli 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| filesense_cli-0.1.4.tar.gz | 37.9 kB | With provenance | Details for filesense_cli-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| filesense_cli-0.1.4-py3-none-any.whl | Python 3 | none | any | Details for filesense_cli-0.1.4-py3-none-any.whl |
Total release size: 63.9 kB
Release files / filesense_cli-0.1.4.tar.gz
| Download URL | filesense_cli-0.1.4.tar.gz |
|---|---|
| Size | 37.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
69c781b9baf043efe00f69faeb45b8ba3e1224bbc740d40a416b36d9e5639ed0
|
|
BLAKE2b-256 checksum How to use checksums |
edff1ee1c43230c2347b21dc0d26035ca59497d10b2b53a84c2a2bbd0b614b09
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.
Transparency logRelease files / filesense_cli-0.1.4-py3-none-any.whl
| Download URL | filesense_cli-0.1.4-py3-none-any.whl |
|---|---|
| Size | 26.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
156ced4872ad040a4ac3d7a3f6168af5f3e025831141744418dd29d04bb0321d
|
|
BLAKE2b-256 checksum How to use checksums |
0ed7fc77ab55a635213eeea100a0be375df837b7f2a0bdc0fad155fc5b4e752f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.
Transparency logSecurity for filesense-cli 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for filesense_cli-0.1.4.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 17, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.