tencent-cos-cli 2.2.1

Command line interface for Tencent Cloud Object Storage

Tencent COS CLI

A powerful command-line interface for Tencent Cloud Object Storage (COS), designed with a similar experience to AWS CLI.

Features

• 🚀 Easy to Use: Intuitive commands similar to AWS CLI
• 🔐 Secure: Support for STS temporary credentials and role assumption
• ⚡ Fast: Multipart uploads and parallel transfers
• 🎨 Beautiful: Rich progress bars and formatted output
• 🔧 Flexible: Multiple profiles, regions, and output formats
• 📦 Complete: Upload, download, sync, and manage buckets

Installation

From PyPI (Recommended)

pip install tencent-cos-cli

That's it! The cos command will be available in your PATH.

From Source (Development)

Quick Install (with new venv)

git clone https://github.com/sszhu/coscli.git
cd coscli
./install.sh

This will:

1. Install uv (if not present)
2. Create a virtual environment
3. Install COS CLI and dependencies

Then activate:

source .venv/bin/activate

Install in Current Environment

If you're already in a conda environment or existing venv:

./install.sh --current

This installs COS CLI in your current Python environment without creating a new venv

Manual Installation

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create venv and install
uv venv
source .venv/bin/activate
uv pip install -e .

Requirements

• Python 3.8+
• uv package manager
• Tencent Cloud Account with COS access

Progress Bars & Non-TTY Environments

• Progress is enabled by default in interactive terminals.
• In non-TTY environments (e.g., CI), progress is automatically disabled.
• You can force-disable progress for scripts with --no-progress.

Troubleshooting Installation

If you encounter SSL certificate errors during installation (common in corporate networks):

# The install.sh script handles this automatically, but if needed:
curl -LsSfk https://astral.sh/uv/install.sh | sh

SSL Certificate Issues?

# Run diagnostic tool to identify the issue
python -m cos.tools.diagnose_ssl

📖 More details:

• docs/UV_GUIDE.md
• docs/SSL_TROUBLESHOOTING.md - Comprehensive SSL troubleshooting

Quick Start

1. Configure Credentials

Run the interactive configuration:

cos configure

You'll be prompted for:

• Secret ID: Your Tencent Cloud secret ID
• Secret Key: Your Tencent Cloud secret key
• Assume Role ARN: (Optional) Role to assume
• Default Region: Default region (e.g., ap-shanghai)
• Default Bucket: (Optional) Default bucket name
• Default Prefix: (Optional) Default prefix
• Output Format: json, table, or text

Generate Temporary Credentials

For testing or temporary access:

# Generate 2-hour temporary token (basic)
cos token

# Generate 1-hour token
cos token --duration 3600

# Generate prefix-restricted token (scoped access)
cos token --bucket mybucket-1234567890 --prefix "data/uploads"

# Generate read-only token for specific prefix
cos token --bucket mybucket-1234567890 --prefix "reports" --read-only

# Specify custom region and actions
cos token --bucket mybucket-1234567890 --region ap-shanghai \
          --action GetObject --action PutObject

# Export as environment variables for immediate use
cos token --output env > temp_creds.sh
source temp_creds.sh

📖 More details:

• docs/TOKEN_USAGE_GUIDE.md - Token management and CI/CD integration
• docs/STS_PREFIX_ACCESS_GUIDE.md - Prefix-restricted access

2. Basic Commands

List Buckets

cos ls

List Objects in Bucket

cos ls cos://my-bucket/
cos ls cos://my-bucket/folder/ --recursive

Upload Files

# Upload single file
cos cp local-file.txt cos://my-bucket/remote-file.txt

# Upload directory
cos cp ./local-dir/ cos://my-bucket/remote-dir/ --recursive

# Parallel upload (4 workers) with aggregated progress
cos cp ./local-dir/ cos://my-bucket/remote-dir/ --recursive --concurrency 4

Download Files

# Download single file
cos cp cos://my-bucket/file.txt ./local-file.txt

# Download directory
cos cp cos://my-bucket/folder/ ./local-folder/ --recursive

# Parallel download (8 workers) with aggregated progress
cos cp cos://my-bucket/folder/ ./local-folder/ --recursive --concurrency 8

# Use larger parts (64MB) and more retries for unstable networks
cos cp cos://my-bucket/large.tar.gz ./ --part-size 64MB --max-retries 5 --retry-backoff 1.0 --retry-backoff-max 10.0

# Disable resumption explicitly
cos cp cos://my-bucket/large.tar.gz ./ --no-resume

Copy Between Buckets

cos cp cos://bucket1/file.txt cos://bucket2/file.txt

Delete Objects

# Delete single object
cos rm cos://my-bucket/file.txt

# Delete multiple objects
cos rm cos://my-bucket/folder/ --recursive

Move/Rename Objects

# Rename object
cos mv cos://bucket/old-name.txt cos://bucket/new-name.txt

# Move directory
cos mv cos://bucket/old-folder/ cos://bucket/new-folder/ --recursive

# Move between buckets
cos mv cos://bucket1/file.txt cos://bucket2/file.txt

# Use larger parts and retries for local->COS move
cos mv ./bigfile.bin cos://bucket/bigfile.bin --part-size 64MB --max-retries 5 --retry-backoff 1.0 --retry-backoff-max 10.0

Synchronize Directories

# Sync local to COS (upload)
cos sync ./local-dir/ cos://bucket/remote-dir/

# Sync COS to local (download)
cos sync cos://bucket/remote-dir/ ./local-dir/

# Sync with delete (exact mirror)
cos sync ./local-dir/ cos://bucket/remote-dir/ --delete

# Preview sync without making changes
cos sync ./local-dir/ cos://bucket/remote-dir/ --dryrun

# Fast sync (compare by size only)
cos sync ./local-dir/ cos://bucket/remote-dir/ --size-only

# Apply part-size and retry settings; use resumable ranged downloads
cos sync cos://bucket/remote-dir/ ./local-dir/ --part-size 64MB --max-retries 5 --retry-backoff 1.0 --retry-backoff-max 10.0 --resume

Generate Presigned URLs

# Generate URL for download (default 1 hour)
cos presign cos://bucket/file.txt

# Custom expiration (2 hours)
cos presign cos://bucket/file.txt --expires-in 7200

# Generate upload URL
cos presign cos://bucket/file.txt --method PUT

# Generate short-lived URL (5 minutes)
cos presign cos://bucket/file.txt -e 300

Create/Delete Buckets

# Create bucket
cos mb cos://my-new-bucket

# Delete bucket
cos rb cos://my-bucket

# Delete bucket with all contents
cos rb cos://my-bucket --force

Advanced Usage

Multiple Profiles

Configure multiple profiles for different accounts or environments:

# Configure production profile
cos configure --profile production

# Use production profile
cos ls --profile production

# Set profile in environment
export COS_PROFILE=production
cos ls

Output Formats

Choose from three output formats:

# Table format (default)
cos ls cos://bucket/ --output table

# JSON format
cos ls cos://bucket/ --output json

# Text format (for scripting)
cos ls cos://bucket/ --output text

Region Override

# Override default region
cos ls cos://bucket/ --region ap-beijing

# Set region in environment
export COS_REGION=ap-beijing

Filtering

# Include specific patterns
cos cp ./dir/ cos://bucket/ --recursive --include "*.txt"

# Exclude specific patterns
cos cp ./dir/ cos://bucket/ --recursive --exclude "*.log"

Progress Control

# Disable progress bars
cos cp large-file.iso cos://bucket/ --no-progress

# Quiet mode
cos cp file.txt cos://bucket/ --quiet

Transfer Tuning

Tune performance and resilience of uploads/downloads across commands:

• Cheat sheet

Flag	Applies To	Default	Notes
--concurrency	cp -r	4	Parallel workers for recursive transfers
--part-size	cp, mv (local→COS), sync	8MB	Per-part size for multipart uploads and ranged downloads
--max-retries	cp, mv (local→COS), sync	3	Retries per part/range on transient errors
--retry-backoff	cp, mv, sync	0.5s	Initial backoff (exponential)
--retry-backoff-max	cp, mv, sync	5.0s	Max backoff cap
--resume/--no-resume	cp downloads, sync downloads	--resume	Resume ranged downloads from partial files
--no-progress	all	off in TTY	Auto-disabled in non‑TTY (e.g., CI)

• --concurrency: Parallel workers for recursive cp operations.
• --part-size: Size of each part/chunk for multipart uploads and ranged downloads. Accepts B, KB, MB, GB (e.g., 8MB, 64MB). Default: 8MB.
• --max-retries: Max retries per part/range for network or transient errors. Default: 3.
• --retry-backoff: Initial backoff seconds between retries (exponential). Default: 0.5.
• --retry-backoff-max: Maximum backoff seconds cap. Default: 5.0.
• --resume/--no-resume: Enable resumable ranged downloads (continue from partial files). Default: enabled.
• --no-progress: Disable progress bars. Progress is auto-disabled in non‑TTY (e.g., CI).

Command-specific behavior

• cp:
  • Uploads: --part-size, retry/backoff apply to single-file uploads (multipart with progress). Recursive uploads use --concurrency.
  • Downloads: uses ranged downloads with --part-size, retry/backoff, and --resume when progress is enabled or not explicitly disabled by non‑TTY.
• mv:
  • Local → COS: with progress enabled, uses multipart upload honoring --part-size and retry/backoff. With --no-progress, uses the simple SDK path (no multipart progress), keeping behavior consistent with scripts/tests.
  • COS → COS: server-side copy; tuning flags don’t apply.
• sync:
  • Local → COS uploads: with progress enabled, multipart upload honors --part-size and retry/backoff; with --no-progress, uses simple upload.
  • COS → Local downloads: with progress enabled, ranged download honors --part-size, retry/backoff, and --resume; with --no-progress, uses simple download.

Tips

• Larger --part-size reduces request overhead but increases memory per part and retry cost. Smaller parts increase concurrency headroom but add overhead.
• Increase --max-retries and --retry-backoff on unstable networks.
• Disable --resume to force fresh downloads when you don’t want to reuse partial files.

Configuration

Configuration Files

Configuration is stored in ~/.cos/:

~/.cos/
├── config          # Configuration settings
└── credentials     # Credentials (secret keys)

Configuration File Format

~/.cos/config:

[default]
region = ap-shanghai
output = json

[profile production]
region = ap-beijing
output = table

Credentials File Format

~/.cos/credentials:

[default]
secret_id = AKIDxxxxx
secret_key = xxxxxxxxx
assume_role = qcs::cam::uin/xxx:roleName/xxx

[profile production]
secret_id = AKIDxxxxx
secret_key = xxxxxxxxx

Environment Variables

Override configuration with environment variables:

export COS_SECRET_ID=AKIDxxxxx
export COS_SECRET_KEY=xxxxxxxxx
export COS_REGION=ap-shanghai
export COS_ASSUME_ROLE=qcs::cam::uin/xxx:roleName/xxx
export COS_OUTPUT=json
export COS_PROFILE=production

Credential Precedence

When multiple credential sources are configured, the CLI follows a specific precedence order:

1. Environment Temporary Token (COS_TOKEN + COS_SECRET_ID + COS_SECRET_KEY)
2. Config File Temporary Token (via cos configure import-token)
3. STS via Assume Role (assume_role in config file)
4. Permanent Credentials (config file or environment variables)

Important: When COS_TOKEN is set in environment, all config file settings (including assume_role) are ignored to prevent conflicts.

📖 See Credential Precedence Guide for detailed rules and troubleshooting.

Command Reference

Global Options

--profile TEXT              Use specific profile
--region TEXT               Override default region
--output [json|table|text]  Output format
--endpoint-url TEXT         Custom endpoint URL
--no-verify-ssl            Skip SSL verification
--debug                    Enable debug mode
--quiet                    Suppress output

Commands

Command	Description
configure	Configure COS CLI settings
ls	List buckets or objects
cp	Copy files to/from/within COS
mv	Move or rename objects
sync	Synchronize directories between local and COS
rm	Remove objects
mb	Make (create) bucket
rb	Remove bucket
presign	Generate presigned URLs for temporary access
token	Generate temporary STS credentials
lifecycle	Manage bucket lifecycle policies
policy	Manage bucket access policies
cors	Configure CORS settings
versioning	Manage bucket versioning

Advanced Commands

Bucket Lifecycle Management

Control automatic transitions and expiration of objects:

# View lifecycle rules
cos lifecycle get cos://my-bucket

# Set lifecycle rules from JSON file
cos lifecycle put cos://my-bucket --config lifecycle.json

# Delete lifecycle configuration
cos lifecycle delete cos://my-bucket

Example lifecycle.json:

{
  "Rule": [
    {
      "ID": "Delete old logs",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Expiration": {"Days": 30}
    },
    {
      "ID": "Archive to ARCHIVE",
      "Status": "Enabled",
      "Filter": {"Prefix": "archive/"},
      "Transition": {
        "Days": 90,
        "StorageClass": "ARCHIVE"
      }
    }
  ]
}

Bucket Policy Management

Control access permissions at bucket level:

# View current policy
cos policy get cos://my-bucket

# Set policy from JSON file
cos policy put cos://my-bucket --policy policy.json

# Delete policy
cos policy delete cos://my-bucket

Example policy.json:

{
  "version": "2.0",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"qcs": ["qcs::cam::uin/100000000001:uin/100000000001"]},
      "Action": ["name/cos:GetObject"],
      "Resource": ["qcs::cos:ap-shanghai:uid/1250000000:my-bucket/*"]
    }
  ]
}

CORS Configuration

Configure Cross-Origin Resource Sharing:

# View CORS rules
cos cors get cos://my-bucket

# Set CORS rules from JSON file
cos cors put cos://my-bucket --config cors.json

# Delete CORS configuration
cos cors delete cos://my-bucket

Example cors.json:

{
  "CORSRule": [
    {
      "ID": "Allow all origins",
      "AllowedOrigin": ["*"],
      "AllowedMethod": ["GET", "PUT", "POST", "DELETE", "HEAD"],
      "AllowedHeader": ["*"],
      "ExposeHeader": ["ETag", "Content-Length"],
      "MaxAgeSeconds": 3600
    }
  ]
}

Bucket Versioning

Enable object versioning for protection:

# Check versioning status
cos versioning get cos://my-bucket

# Enable versioning
cos versioning enable cos://my-bucket

# Suspend versioning
cos versioning suspend cos://my-bucket

Pattern Matching

Filter files during transfer operations:

# Upload only Python files
cos cp ./project/ cos://bucket/code/ -r --include "*.py"

# Upload all except tests
cos cp ./project/ cos://bucket/code/ -r --exclude "test_*"

# Complex patterns: include source files, exclude tests and cache
cos cp ./project/ cos://bucket/backup/ -r \
  --include "*.py" --include "*.js" \
  --exclude "test_*" --exclude "__pycache__"

# Download only log files
cos cp cos://bucket/logs/ ./local-logs/ -r --include "*.log"

Checksum Verification

Ensure data integrity with MD5 checksums:

# Sync with checksum verification
cos sync ./local/ cos://bucket/remote/ --checksum

# Even if timestamps differ, skip files with matching checksums
cos sync ./local/ cos://bucket/remote/ --checksum --size-only

Examples

Backup Local Directory to COS

cos cp ./my-data/ cos://my-backup-bucket/$(date +%Y%m%d)/ --recursive

Download Latest Files

cos ls cos://bucket/logs/ --output text | tail -5 | while read file; do
  cos cp cos://bucket/$file ./
done

Sync Directory

# Upload only changed files
cos sync ./local-dir/ cos://bucket/backup/ --size-only

# Exact mirror (delete files not in source)
cos sync ./local-dir/ cos://bucket/backup/ --delete

Share Files with Presigned URLs

# Generate a download link valid for 24 hours
cos presign cos://bucket/document.pdf --expires-in 86400

# Share via curl
URL=$(cos presign cos://bucket/file.txt)
echo "Download: $URL"

Bulk Delete Old Files

cos ls cos://bucket/archive/ --output text | while read file; do
  cos rm cos://bucket/$file
done

Troubleshooting

Authentication Errors

If you get authentication errors:

1. Verify credentials: cos configure list
2. Check credential file permissions: ls -la ~/.cos/
3. Try with debug mode: cos ls --debug

SSL Verification Errors

If you encounter SSL errors:

cos ls --no-verify-ssl

Large File Uploads

For files larger than 5MB, multipart upload is automatic. For very large files:

• Ensure stable network connection
• Use --no-progress for background operations
• Check available disk space

Development

Running Tests

pytest tests/

Code Style

black cos/
flake8 cos/
mypy cos/

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new features
4. Submit a pull request

License

MIT License - see LICENSE file for details

Documentation

• Quick Reference: docs/QUICK_REFERENCE.md
• Development Plan: docs/COS_CLI_DEVELOPMENT_PLAN.md
• Implementation Summary (v1.1.0): docs/IMPLEMENTATION_SUMMARY.md
• Implementation Complete (v2.0.0): docs/IMPLEMENTATION_COMPLETE_V2.md
• Final Verification Report: docs/VERIFICATION_COMPLETE.md
• Release Notes (v2.0.0): docs/RELEASE_NOTES_2.0.0.md
• Full Index: docs/INDEX.md

Changelog

See CHANGELOG.md for version history.

License

This project is licensed under the MIT License - see the LICENSE file for details.

What This Means

✅ You can:

• Use this software for commercial purposes
• Modify and distribute the software
• Use it privately
• Sublicense it

⚠️ You must:

• Include the original copyright notice
• Include the MIT License text

❌ Limitations:

• No warranty provided
• Authors are not liable for any claims

Credits

Built with:

• uv - Ultra-fast package installer
• Click - CLI framework
• Rich - Beautiful terminal output
• Tencent Cloud SDK - Official SDK

Author

Shanshan Zhu

Contributions are welcome! Please feel free to submit a Pull Request.