Python module to download objects in an S3 bucket.
Project description
Versions Supported
Language Stats
Repo Stats
Activity
Build Status
S3 Download
Python module to download objects in an S3 bucket.
Installation
pip install s3-downloader
Usage
Download objects in parallel
import s3
if __name__ == '__main__':
wrapper = s3.Downloader(bucket_name='BUCKET_NAME')
wrapper.run_in_parallel(threads=10) # Defaults to 5
Download objects in sequence
import s3
if __name__ == '__main__':
wrapper = s3.Downloader(bucket_name='BUCKET_NAME')
wrapper.run()
Mandatory arg
- bucket_name - Name of the s3 bucket.
Optional kwargs
- prefix - Prefix to filter the objects based on their path. Defaults to
None
- logger - Bring your own custom pre-configured logger. Defaults to on-screen logging.
- download_dir - Name/path of the directory where the objects have to be stored.
Defaults to
bucket_name
at current working directory. - region_name - AWS region name. Defaults to the env var
AWS_DEFAULT_REGION
- profile_name - AWS profile name. Defaults to the env var
PROFILE_NAME
- aws_access_key_id - AWS access key ID. Defaults to the env var
AWS_ACCESS_KEY_ID
- aws_secret_access_key - AWS secret access key. Defaults to the env var
AWS_SECRET_ACCESS_KEY
AWS values are loaded from env vars or the default config at
~/.aws/config
/~/.aws/credentials
Coding Standards
Docstring format: Google
Styling conventions: PEP 8
Clean code with pre-commit hooks: flake8
and
isort
Release Notes
Requirement
python -m pip install gitverse
Usage
gitverse-release reverse -f release_notes.rst -t 'Release Notes'
Linting
pre-commit
will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL
markdown files (including Wiki pages)
Requirement
pip install sphinx==5.1.1 pre-commit recommonmark
Usage
pre-commit run --all-files
Pypi Package
https://pypi.org/project/s3-downloader/
License & copyright
© Vignesh Rao
Licensed under the MIT License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for s3_downloader-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5989f288c15c18db5a2bb3f9d6b22b84a21eb4e2636807f052a7a2df436b6ad |
|
MD5 | 025d6153f285339f738cacae08a20d30 |
|
BLAKE2b-256 | e85241ded5c43f09f963387fcc054f1c8a609d0838a98485beb19e6cf57b72df |