gmail-archiver-cli 1.4.2

A CLI tool to archive old Gmail messages to local mbox files with validation and safe deletion

GMailArchiver

A professional-grade email archival, search, and management solution for Gmail - Archive, compress, search, extract, and maintain your email history with confidence.

🎉 What's New in v1.3.2 - Critical Bug Fix

Version 1.3.2 fixes a critical bug that caused archiving failures:

• 🐛 Fixed UNIQUE Constraint Errors - Messages with duplicate RFC Message-IDs (same email in multiple folders) no longer cause database errors
• 🛡️ Improved Duplicate Handling - Duplicates are now skipped gracefully with proper logging
• ✅ Enhanced Stability - Eliminates orphaned messages in mbox files

Recent Major Features (v1.2.0)

• 🎨 Unified Rich Output - Beautiful terminal output with progress bars, ETA, and rate tracking
• 📤 Message Extraction - Retrieve messages from search results
• 🏥 Health Check Command - One command to check everything
• ⏰ Automated Scheduling - Set up periodic health checks
• 🗜️ Post-hoc Compression - Compress existing archives anytime
• 🩺 Doctor Command - Comprehensive diagnostics
• 🔍 Search Enhancements - Preview and interactive modes
• 📊 JSON Mode - All commands support --json for scripting

See full changelog

Why Gmail Archiver?

Gmail offers 15GB of free storage shared across Google services, but that space fills up quickly with years of emails, attachments, and files. While Gmail provides basic search and labels, it lacks:

• Local backup and control: Your emails are only in Google's cloud
• Long-term archival: No built-in way to archive and compress old emails while keeping them searchable
• Data portability: Difficult to export and search emails outside Gmail
• Storage optimization: No automatic compression or deduplication
• Fast local search: Gmail search can be slow for large mailboxes

Gmail Archiver solves these problems by providing a professional-grade archival solution that:

1. Archives old emails to portable mbox files (industry standard format)
2. Searches archived emails with Gmail-style syntax (faster than Gmail itself!)
3. Extracts individual messages from compressed archives
4. Compresses archives with modern algorithms (zstd, lzma, gzip)
5. Validates archives before deletion with multi-layer verification
6. Manages your email history with deduplication and consolidation
7. Automates maintenance with scheduled health checks
8. Protects your data with atomic transactions and safe deletion workflows

Key Benefits

• Reclaim Gmail storage: Archive old emails and safely delete them from Gmail
• Keep emails searchable: Lightning-fast full-text search (0.85ms for 1000 messages)
• Extract on demand: Retrieve individual messages from compressed archives
• Maintain data sovereignty: Your emails, your local storage, your control
• Automate maintenance: Set-and-forget health checks
• Future-proof format: mbox is a 40+ year old standard supported by all email clients
• Production-ready: 989 automated tests, 93% code coverage, strict type safety

✨ Core Features

📥 Archiving & Deletion

• Smart Archiving: Archive emails older than a specified threshold (e.g., "3y", "6m", "30d")
• Incremental Mode: Skip already-archived messages for efficient recurring runs
• Safe Deletion Workflow: Archive-only (default) → Trash (reversible) → Permanent (confirmed)
• Batch Operations: Efficient API usage with automatic rate limiting
• Progress Tracking: Real-time progress bars with ETA and processing rate

🔍 Search & Retrieval

• Full-Text Search (FTS5): Gmail-style query syntax with BM25 ranking
• Lightning Fast: 0.85ms for 1000 messages (118x faster than target)
• Message Extraction: Extract individual messages by ID or from search results
• Interactive Search: Browse search results with a menu interface
• Preview Mode: See message snippets in search results
• JSON Output: All commands support --json for scripting

🗜️ Compression & Storage

• Modern Compression: zstd (fastest), lzma (smallest), gzip (compatible)
• Post-hoc Compression: Compress existing archives anytime
• Transparent Decompression: Read compressed archives without extraction
• Smart Deduplication: Remove duplicates across archives (100% precision)
• Archive Consolidation: Merge multiple archives with automatic sorting

🛡️ Safety & Validation

• Multi-Layer Validation: Message count, database cross-check, content integrity, spot-checks
• Unified Health Check: One command checks database, archives, auth, performance
• Auto-Repair: Automatic database repair with rollback support
• Atomic Operations: All writes are transactional (succeed or rollback)
• Audit Trail: Complete history of all operations

⚙️ Automation & Maintenance

• Scheduled Health Checks: Platform-native cron/Task Scheduler integration
• Automatic Migration: v1.0 → v1.1 → v1.2 schema upgrades with backup
• Comprehensive Diagnostics: Doctor command analyzes everything
• Auto-Verification: Optional validation after import/consolidate/dedupe
• Performance Metrics: Track search latency, database size, vacuum status

📦 Installation

Prerequisites

• Python 3.14+ (Download here)
• Gmail Account with email you want to archive

Note: OAuth2 credentials are bundled with the application. No manual Google Cloud setup required!

Install from PyPI (Recommended)

pip install gmail-archiver-cli

Or use pipx for isolated installation:

pipx install gmail-archiver-cli

Verify Installation

gmailarchiver --version
gmailarchiver --help

🚀 Quick Start

First Run - OAuth2 Authorization

On first run, Gmail Archiver will automatically:

1. Open your browser to Google's authorization page
2. Ask you to sign in with your Google Account
3. Request permission to access Gmail
4. Save an authorization token to ~/.config/gmailarchiver/token.json

Basic Workflow

# 1. Preview what will be archived (dry run)
gmailarchiver archive 3y --dry-run

# 2. Archive emails older than 3 years with compression
gmailarchiver archive 3y --compress zstd
# → Creates: archive_20250123.mbox.zst

# Or use exact dates (v1.3.0+)
gmailarchiver archive 2024-01-01 --compress zstd
# → Archives all emails before January 1, 2024

# 3. Validate the archive
gmailarchiver validate archive_20250123.mbox.zst

# 4. Search your archives
gmailarchiver search "from:[e-mail] meeting"

# 5. Extract a message
gmailarchiver extract msg_123abc --output message.eml

# 6. Check overall health
gmailarchiver check

# 7. (Optional) Delete from Gmail after verification
gmailarchiver archive 3y --trash  # Reversible (30 days)

📖 Command Reference

For complete command documentation with all options, see docs/USAGE.md.

Quick Reference

Category	Commands
Archiving	archive, import, consolidate, compress
Search	search, extract
Health	check, doctor, verify-integrity, repair
Maintenance	dedupe, status, schedule
Auth	auth-reset, migrate, rollback

Key Commands

# Archive emails older than 3 years with compression
gmailarchiver archive 3y --compress zstd

# Search archived messages
gmailarchiver search "from:[e-mail] subject:meeting"

# Extract a specific message
gmailarchiver extract msg_123abc --output message.eml

# Run all health checks
gmailarchiver check

# Show status with database info
gmailarchiver status --verbose

# Full diagnostics
gmailarchiver doctor

All commands support --json for scripting and --help for detailed options.

📊 Performance

Operation	Dataset	Time	Rate
Search (metadata)	1,000 messages	0.85ms	1.2M msg/s
Search (full-text)	1,000 messages	45ms	22K msg/s
Import	10,000 messages	<1s	10K+ msg/s
Consolidate	10,000 messages	3.57s	2.8K msg/s
Extract	Single message	<10ms	N/A

🔒 Security & Privacy

• OAuth2 Flow: Industry-standard authentication
• Scopes: Minimum required permissions (gmail.modify for deletion)
• Token Storage: XDG-compliant paths (~/.config/gmailarchiver/)
• Local Storage: All data stored locally, no cloud dependencies
• Audit Trail: Complete operation history in database
• Safe Deletion: Trash-first workflow with 30-day recovery window

📚 Additional Documentation

• Usage Guide - Complete command reference with all options
• Architecture Documentation - System design and technical details
• Migration Guide - Upgrading from v1.0.x
• Contributing Guide - Development setup and guidelines
• Changelog - Version history and release notes

📜 Version History

v1.3.2 (2025-11-24) - Critical Bug Fix

Bug Fixes:

• Fixed UNIQUE constraint failures during archiving (messages with duplicate RFC Message-IDs)
• Improved duplicate detection to check rfc_message_id before writing to mbox
• Eliminated orphaned messages in mbox files

Quality: 1072 tests, 93% coverage maintained

v1.3.1 (2025-11-24) - Live Layout Infrastructure

Internal Features:

• Added LogBuffer, SessionLogger, and LiveLayoutContext for flicker-free progress tracking
• Enhanced OutputManager with live layout support

Quality: 1071 tests, 93% coverage

v1.2.0 (2025-11-23) - Ergonomics & Automation

Major Features:

• Unified OutputManager with Rich output and JSON mode
• 5 new commands: extract, check, schedule, compress, doctor
• Progress bars with ETA and rate tracking
• Search enhancements (--with-preview, --interactive)
• Auto-verification flags (--auto-verify)
• Cleanup options (--remove-sources)

Test Coverage: 989 tests, 93% coverage

v1.1.0 (2025-11-15) - Search & Management

Major Features:

• FTS5 full-text search with Gmail-style syntax
• Import existing archives (10K+ msg/s)
• Deduplication (100% precision)
• Archive consolidation
• Enhanced database schema (v1.1)

Test Coverage: 650 tests, 96% coverage

v1.0.x - Initial Releases

• Core archiving functionality
• Multi-layer validation
• Safe deletion workflows
• Compression support (gzip, lzma, zstd)

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for:

• Development environment setup
• Testing guidelines
• Code quality standards
• Pull request process

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🙏 Acknowledgments

• Built with Python 3.14 and modern type checking
• Uses Gmail API for email access
• Rich library for beautiful terminal output
• SQLite FTS5 for full-text search
• Python mbox for email archive handling

📧 Support

• Issues: GitHub Issues
• Documentation: docs/
• Changelog: CHANGELOG.md

---

Made with ❤️ for email power users who value privacy, control, and local data ownership.