dedrive 0.1.1

🔍 Find and manage duplicate files in Google Drive using MD5 checksums 🗑️

Features · Quick Start · Configuration · Usage

Features

• Fast MD5-based detection - Identifies duplicates by comparing file checksums, not just names
• Two interfaces - CLI for quick scans, Web UI for interactive review with file previews
• Non-destructive - Moves duplicates to /_dupes folder instead of deleting them
• Preserves structure - Original folder hierarchy is maintained under the dupes folder
• Resumable sessions - Decisions auto-save and persist across sessions
• Flexible filtering - Scan specific paths and exclude folders from analysis
• Multi-account profiles - Target different Google Drive accounts with named profiles

Quick Start

# Install with uv
uv sync

# Launch the web UI
uv run main.py

First run: A browser window will open for Google OAuth authentication. Grant access to your Google Drive.

Installation

Prerequisites

• Python 3.10+
• uv package manager
• Google Cloud OAuth credentials (setup guide)

Google Cloud Setup

1. Go to Google Cloud Console
2. Create a project (or select existing)
3. Enable the Google Drive API
4. Create OAuth 2.0 Client ID (choose "Desktop app")
5. Download the JSON file and save as credentials.json in the project root

Usage

CLI Tool

# Scan entire drive
uv run main.py

# Scan specific folder
uv run main.py --path "/Photos"

# Exclude folders
uv run main.py --exclude "/Backup/Old" --exclude "/tmp"

# Custom output location
uv run main.py --output results.csv

# Validate credentials
uv run main.py --validate

# Debug logging
uv run main.py --verbose --log-file debug.log

# Profiles (multiple Google accounts)
uv run main.py --init-profile work
uv run main.py --list-profiles
uv run main.py --profile work

Web UI

uv run main.py

The web interface provides three tabs:

Note: PDF preview requires poppler: brew install poppler (macOS)

Moving Duplicates

Instead of deleting, duplicates are moved to /_dupes at Drive root:

/Photos/2024/IMG.jpg  →  /_dupes/Photos/2024/IMG.jpg

1. Scan - Find duplicates
2. Review - Mark which files to keep
3. Preview - Dry run to see what would move
4. Execute - Move duplicates to /_dupes

Configuration

Settings can be configured via environment variables, config.json, or CLI arguments.

Precedence: CLI > Profile config.yaml > Environment > Config file > Defaults

Environment Variables

Config File

Create config.json in the project root:

{
  "credentials_path": "~/.config/dedrive/credentials.json",
  "output_dir": "~/.local/share/dedrive",
  "dupes_folder": "/_dupes",
  "batch_size": 100,
  "exclude_paths": ["/Backup/Old", "/tmp"]
}

Profiles

Use profiles to manage multiple Google Drive accounts:

# Create a profile
uv run main.py --init-profile work

# Copy credentials into the profile
cp ~/Downloads/credentials.json profiles/work/

# Use the profile
uv run main.py --profile work

Each profile stores its own credentials.json, token.json, config.yaml, and .output/ under profiles/<name>/.

Output Files

How It Works

1. OAuth authentication - Cached in token.json after first login
2. Single API call - Fetches all files with MD5 metadata in one paginated request
3. In-memory path resolution - Builds paths from parent IDs with memoization
4. MD5 grouping - Groups files by checksum to identify duplicates
5. Size validation - Files with same MD5 but different sizes flagged as "uncertain"

Note: Google Workspace files (Docs, Sheets, Slides) are skipped as they don't have MD5 checksums.

Re-authentication

If you previously used this tool with read-only access, delete token.json and re-authenticate to grant move permissions.

License

MIT