Skip to main content

Aggregate an S3 folder into a file in another

Project description

Makes a tarball (optionally compressed with gzip) of the contents of an S3 prefix (preserving interior paths and filenames), posts the tarball to a stated S3 URL and optionally deletes the source files.

s3clumper: Aggregate S3 prefixes into tarballs in S3

Usage: s3clumper {{arguments}} {{options}}

Arguments:
  from [text]  S3 URL prefix to clump
  to [text]    S3 URL for target clump

Options:
  -h, --help      Show this help message and exit
  -H, --HELP      Help for all sub-commands
  -c, --check     Don't check for target (may over-write)
  -C, --nocolour  Suppress colours in reports
  -D, --debug     Enable debug logging
  -d, --delete    Don't delete source files
  -q, --quiet     Be quiet, be vewy vewy quiet
  -v, --verbose   Verbose output
  -V, --version   Report installed version
  -z, --compress  Don't compress the target

Example:

$ s3clumper s3://bucket1/data/ s3://bucket2/archive/logs-20170302.tar.gz
Fetching |################################| 55/55
Sending |################################| 100/100
Deleting |################################| 55/55

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for s3clumper, version 0.1.post16
Filename, size & hash File type Python version Upload date
s3clumper-0.1.post16.tar.gz (24.1 kB) View hashes Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page