Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

A tool to get all files from an S3 bucket

Project description


A tool to download all objects from an S3 bucket. Supports filtering keys by a prefix.


-> % sucket --help

  Download all files from a S3 bucket

  Everything from the bucket BUCKET_NAME is downloaded, with an optional key
  filter, specified with PREFIX

  By default the "folder" mode is used, which will keep the bucket "folder
  structure" when downloading.

  The "flat" mode will download all objects into the current folder by
  adding the folder structure into the key name.

  The "keys-only" will completely disregard the folders and put all files in
  the current folder.

  -m, --mode [folder|flat|keys-only]
                                  The structure to download the objects in.
  -y, --yes                       Don't prompt for continuing
  -q, --quiet                     Don't print out any info, assumes --yes
  -s, --semaphores INTEGER        Max number of asynchronous requests to make.
                                  Default: 1000
  --help                          Show this message and exit.


folder flat keys-only


This has only been tested on a fairly limited set of data, but has worked well for ~3500 small files. Need to tweak and experiment with larger files.

Special thanks

This is heavily based on a script that @steinnes wrote some time ago and has been very useful to me for a long time, the name was entirely his idea.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sucket, version 0.3.4
Filename, size File type Python version Upload date Hashes
Filename, size sucket-0.3.4-py3-none-any.whl (5.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size sucket-0.3.4.tar.gz (4.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page