s3hero 1.0.0

A powerful CLI tool to manage S3 buckets across AWS, Cloudflare R2, and S3-compatible services

S3Hero 🦸‍♂️

A powerful CLI tool to manage S3 buckets across AWS S3, Cloudflare R2, and other S3-compatible services.

✨ Features

• 🪣 Full Bucket Management - Create, list, delete, and empty buckets
• 📁 Object Operations - Upload, download, copy, move, and delete files
• 🔄 Sync Support - Sync directories to/from S3 with progress tracking
• 🌐 Multi-Provider Support - AWS S3, Cloudflare R2, MinIO, and any S3-compatible service
• 👤 Profile Management - Multiple profiles for different accounts/providers
• 📊 Rich Output - Beautiful tables, trees, and progress bars
• 🔗 Presigned URLs - Generate temporary access URLs
• 💨 Bulk Operations - Empty buckets, recursive deletes, and more

📦 Installation

Quick Install (Linux/macOS)

curl -fsSL https://raw.githubusercontent.com/aravindgopall/s3hero/main/scripts/install.sh | bash

Quick Install (Windows)

Open PowerShell as Administrator and run:

Set-ExecutionPolicy Bypass -Scope Process -Force
irm https://raw.githubusercontent.com/aravindgopall/s3hero/main/scripts/install.ps1 | iex

Install with pip

# Create and activate a virtual environment (recommended)
python3 -m venv ~/.s3hero/venv
source ~/.s3hero/venv/bin/activate  # On Windows: ~/.s3hero/venv/Scripts/activate

# Install s3hero
pip install s3hero

Install from Source

# Clone the repository
git clone https://github.com/aravindgopall/s3hero.git
cd s3hero

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in development mode
pip install -e .

Install via Homebrew (macOS/Linux)

# Add the tap
brew tap aravindgopall/tap

# Install s3hero
brew install s3hero

Or install directly from the formula:

brew install --build-from-source https://raw.githubusercontent.com/aravindgopall/s3hero/main/scripts/s3hero.rb

🚀 Quick Start

1. Configure Your First Profile

s3hero configure add

This will guide you through setting up your S3 connection:

S3Hero Profile Setup

Profile name [default]: my-aws
Select S3 Provider:
  1. AWS S3
  2. Cloudflare R2
  3. Other S3-Compatible
Provider [1]: 1
Access Key ID: AKIA...
Secret Access Key: ********
AWS Region [us-east-1]: us-west-2

2. List Your Buckets

s3hero bucket list

3. Upload Files

# Upload a single file
s3hero cp myfile.txt s3://my-bucket/myfile.txt

# Upload a directory
s3hero cp -r ./my-folder/ s3://my-bucket/folder/

📖 Usage Guide

Profile Management

# Add a new profile interactively
s3hero configure add

# Add a profile with command-line options
s3hero configure add \
  --name production \
  --provider aws \
  --access-key AKIA... \
  --secret-key ... \
  --region us-east-1 \
  --default

# List all profiles
s3hero configure list

# Show profile details (secrets are hidden)
s3hero configure show my-profile

# Set default profile
s3hero configure default my-profile

# Remove a profile
s3hero configure remove old-profile

Provider-Specific Setup

AWS S3

s3hero configure add \
  --name aws-prod \
  --provider aws \
  --access-key YOUR_ACCESS_KEY \
  --secret-key YOUR_SECRET_KEY \
  --region us-east-1

Cloudflare R2

s3hero configure add \
  --name cf-r2 \
  --provider cloudflare_r2 \
  --access-key YOUR_R2_ACCESS_KEY \
  --secret-key YOUR_R2_SECRET_KEY \
  --account-id YOUR_CLOUDFLARE_ACCOUNT_ID

Other S3-Compatible (MinIO, DigitalOcean Spaces, etc.)

s3hero configure add \
  --name minio-local \
  --provider other \
  --access-key YOUR_ACCESS_KEY \
  --secret-key YOUR_SECRET_KEY \
  --endpoint https://minio.example.com \
  --region us-east-1

Bucket Operations

# List all buckets
s3hero bucket list
s3hero bucket list --json  # JSON output

# Create a bucket
s3hero bucket create my-new-bucket
s3hero bucket create my-bucket --region eu-west-1

# Delete a bucket (must be empty)
s3hero bucket delete my-bucket

# Force delete (empties bucket first)
s3hero bucket delete my-bucket --force

# Empty a bucket (delete all objects)
s3hero bucket empty my-bucket
s3hero bucket empty my-bucket --prefix logs/  # Only delete objects with prefix

# Get bucket size
s3hero bucket size my-bucket

# Check if bucket exists
s3hero bucket exists my-bucket

Object Operations

# List objects
s3hero ls s3://my-bucket
s3hero ls s3://my-bucket/prefix/
s3hero ls s3://my-bucket --tree        # Tree view
s3hero ls s3://my-bucket --limit 100   # Limit results
s3hero ls s3://my-bucket --json        # JSON output

# Upload files
s3hero cp local-file.txt s3://my-bucket/remote-file.txt
s3hero cp -r ./local-dir/ s3://my-bucket/remote-dir/

# Download files
s3hero cp s3://my-bucket/file.txt ./local-file.txt
s3hero cp s3://my-bucket/file.txt ./downloads/

# Copy between S3 locations
s3hero cp s3://bucket-a/file.txt s3://bucket-b/file.txt

# Move/rename objects
s3hero mv s3://my-bucket/old-name.txt s3://my-bucket/new-name.txt

# Delete objects
s3hero rm s3://my-bucket/file.txt
s3hero rm -r s3://my-bucket/folder/     # Recursive delete
s3hero rm -r -y s3://my-bucket/folder/  # Skip confirmation

# View object info
s3hero info s3://my-bucket/file.txt

# View file contents
s3hero cat s3://my-bucket/file.txt

# Generate presigned URL
s3hero presign s3://my-bucket/file.txt
s3hero presign s3://my-bucket/file.txt --expires 7200  # 2 hours

Sync Operations

# Sync local directory to S3
s3hero sync ./local/ s3://my-bucket/remote/

# Sync S3 to local directory
s3hero sync s3://my-bucket/remote/ ./local/

# Sync with delete (mirror)
s3hero sync --delete ./local/ s3://my-bucket/remote/

# Dry run (show what would happen)
s3hero sync --dry-run ./local/ s3://my-bucket/remote/

Using Different Profiles

# Use a specific profile for a command
s3hero -p production bucket list
s3hero --profile staging ls s3://my-bucket

# Set profile via environment variable
export S3HERO_PROFILE=production
s3hero bucket list

Output Formats

# Table format (default)
s3hero bucket list

# JSON format
s3hero bucket list --json
s3hero ls s3://my-bucket --json

# Tree view for objects
s3hero ls s3://my-bucket --tree

🔧 Configuration

Configuration is stored in ~/.s3hero/config.yaml:

default_profile: aws-prod
profiles:
  - name: aws-prod
    provider: aws
    access_key: AKIA...
    secret_key: ...
    region: us-east-1
    
  - name: cf-r2
    provider: cloudflare_r2
    access_key: ...
    secret_key: ...
    account_id: ...
    region: auto
    
  - name: minio
    provider: other
    access_key: ...
    secret_key: ...
    endpoint_url: https://minio.example.com
    region: us-east-1

Environment Variables

Variable	Description
S3HERO_PROFILE	Default profile to use
S3HERO_INSTALL_DIR	Custom installation directory
S3HERO_VENV_DIR	Custom virtual environment directory

🛠️ Development

Setting Up Development Environment

# Clone and enter the repository
git clone https://github.com/aravindgopall/s3hero.git
cd s3hero

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install in development mode with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check src/
black --check src/

# Run type checking
mypy src/

Project Structure

s3hero/
├── src/
│   └── s3hero/
│       ├── __init__.py      # Package metadata
│       ├── cli.py           # CLI commands (Click)
│       ├── client.py        # S3 client wrapper
│       ├── config.py        # Configuration management
│       └── utils.py         # Utilities and formatters
├── scripts/
│   ├── install.sh           # Linux/macOS installer
│   ├── install.ps1          # Windows installer
│   ├── uninstall.sh         # Uninstaller
│   └── s3hero.rb           # Homebrew formula
├── pyproject.toml           # Project configuration
├── requirements.txt         # Dependencies
└── README.md

📦 Publishing to PyPI

# Build the package
pip install build
python -m build

# Upload to PyPI
pip install twine
twine upload dist/*

🍺 Publishing to Homebrew

Create a Homebrew Tap

1. Create a GitHub repository named homebrew-tap
2. Copy scripts/s3hero.rb to Formula/s3hero.rb in your tap
3. Update the formula with correct SHA256 checksums:

# Get SHA256 for release tarball
curl -sL https://github.com/aravindgopall/s3hero/archive/refs/tags/v1.0.0.tar.gz | shasum -a 256

4. Users can then install with:

brew tap aravindgopall/tap
brew install s3hero

🗑️ Uninstallation

Linux/macOS

# Using uninstall script
curl -fsSL https://raw.githubusercontent.com/aravindgopall/s3hero/main/scripts/uninstall.sh | bash

# Or manually
rm -rf ~/.s3hero ~/.local/bin/s3hero

Windows

# Remove installation
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\s3hero"

# Remove from PATH (open System Properties > Environment Variables)

Homebrew

brew uninstall s3hero
brew untap aravindgopall/tap

pip

pip uninstall s3hero

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• boto3 - AWS SDK for Python
• Click - CLI framework
• Rich - Beautiful terminal formatting

---

Made with ❤️ by the S3Hero Team