Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

Add S3 support to dtool

Project description

PyPi package


  • Copy datasets to and from S3 object storage
  • List all the datasets in a S3 bucket
  • Create datasets directly in S3


To install the dtool-S3 package:

pip install dtool-s3


Install the aws client, for details see In short:

pip install awscli --upgrade --user

Configure the credentials using:

aws configure

These are needed for the boto3 library, for more details see


To copy a dataset from local disk (my-dataset) to a S3 bucket (/data_raw) one can use the command below:

dtool copy ./my-dataset s3://data_raw

To list all the datasets in a S3 bucket one can use the command below:

dtool ls s3://data_raw

See the dtool documentation for more detail.



export S3_TEST_BASE_URI=”s3://your-dtool-s3-test-bucket” python develop pytest


$env:S3_TEST_BASE_URI = “s3://your-dtool-s3-test-bucket” python develop pytest

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dtool-s3, version 0.10.0
Filename, size File type Python version Upload date Hashes
Filename, size dtool-s3-0.10.0.tar.gz (8.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page