S3Empty

S3Empty is a Python CLI for conveniently emptying an AWS S3 bucket. It handles versioned and non-versioned S3 buckets.

This tool is useful when you want to delete all objects in a bucket before deleting the bucket itself, which is a common situation often resulting in the following error message:

BucketNotEmpty: The bucket you tried to delete is not empty. You must delete all versions in the bucket.

Installation

pip3 install s3empty

Usage

Run S3Empty with specified bucket name:

s3empty --bucket-name some-bucket

Run S3Empty with a configuration file containing the bucket names:

s3empty --conf-file path/to/some-conf-file.yaml

By default, S3Empty will raise an error if the bucket does not exist. However, there will be scenarios where the S3 bucket you want to empty does not exist or no longer exists. You can use the --allow-inexisting flag to allow inexisting buckets and S3Empty will display a warning message and exits without raising any error:

s3empty --bucket-name some-bucket --allow-inexisting

Show help guide:

s3empty --help

Configuration

You can specify multiple bucket names in S3Empty configuration file and give it a name with .yaml extension, e.g. some-conf-file.yaml :

---
bucket_names:
  - some-bucket-1
  - some-bucket-2

And then call S3Empty:

s3empty --conf-file path/to/some-conf-file.yaml

The configuration file also supports Jinja template where environment variables are available for use. You can give this configuration template a name with .yaml.j2 extension, e.g. some-conf-file.yaml.j2 .

For example, if there is an environment variable ACCOUNT_ID=1234567 , you can specify it in the configuration file:

---
bucket_names:
  - some-{{ env.ACCOUNT_ID }}-bucket-1
  - some-{{ env.ACCOUNT_ID }}-bucket-2

And then call S3Empty:

s3empty --conf-file path/to/some-conf-file.yaml.j2

Permission

Here's an IAM policy with minimum permissions required by S3Empty:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3EmptyPolicy",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketVersioning",
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
            ],
            "Resource": [
                "arn:aws:s3:::some-bucket",
                "arn:aws:s3:::some-bucket/*"
            ]
        }
    ]
}

FAQ

Q: How about using S3 Lifecycle to delete objects in the bucket?

A: S3 Lifecycle modifies the state of the S3 bucket by adding the lifecycle configuration. However, S3Empty aims to only modify the S3 objects without modifying the S3 bucket itself. Other than that, S3 Lifecycle has an expiration delay which means, and I quote, "Amazon S3 might not actually delete these objects until days or even weeks later." S3Empty aims to start deleting the objects immediately.

Colophon

Developer's Guide

Build reports:

• Lint report
• Code complexity report
• Unit tests report
• Test coverage report
• Integration tests report
• API Documentation