Cloud Storage

Cloud Storage is a Python +3.5 package which creates a unified API for the cloud storage services: Amazon Simple Storage Service (S3), Microsoft Azure Storage, Minio Cloud Storage, Rackspace Cloud Files, Google Cloud Storage, and the Local File System.

Cloud Storage is inspired by Apache Libcloud. Advantages to Apache Libcloud Storage are:

Full Python 3 support.
Generate temporary signed URLs for downloading and uploading files.
Support for request and response headers like Content-Disposition.
Pythonic! Iterate through all blobs in containers and all containers in storage using respective objects.

Usage

>>> from cloudstorage.drivers.amazon import S3Driver
>>> storage = S3Driver(key='<my-aws-access-key-id>', secret='<my-aws-secret-access-key>')

>>> container = storage.create_container('avatars')
>>> container.cdn_url
'https://avatars.s3.amazonaws.com/'

>>> avatar_blob = container.upload_blob('/path/my-avatar.png')
>>> avatar_blob.cdn_url
'https://s3.amazonaws.com/avatars/my-avatar.png'

>>> avatar_blob.generate_download_url(expires=3600)
'https://avatars.s3.amazonaws.com/my-avatar.png?'
'AWSAccessKeyId=<my-aws-access-key-id>'
'&Signature=<generated-signature>'
'&Expires=1491849102'

>>> container.generate_upload_url('user-1-avatar.png', expires=3600)
{
    'url': 'https://avatars.s3.amazonaws.com/',
    'fields': {
        'key': 'user-1-avatar.png',
        'AWSAccessKeyId': '<my-aws-access-key-id>',
        'policy': '<generated-policy>',
        'signature': '<generated-signature>'
    }
}

Supported Services

Installation

To install Cloud Storage:

pip install cloudstorage

Also install the storage driver(s) you will be using:

pip install cloudstorage[amazon]
pip install cloudstorage[google]
pip install cloudstorage[local]
pip install cloudstorage[microsoft]
pip install cloudstorage[minio]
pip install cloudstorage[rackspace]

Changelog

1.0.0 (Unreleased)

Features

Start following (Semantic Versioning).

0.11.0 (2021-01-15)

Features

Atomic local file write (#68). Thanks @habibutsu.
New DigitalOcean Spaces Driver (#67). Thanks @RangelReale.

Bugs

Convert bytes to str before saving the json file for Windows Xattr simulator (#66). Thanks @RangelReale!
Handle pathlib.Path properly for Blob.download download and Container.upload_blob (#65). Thanks @sibowsb!
Fix Bob NotFoundError message for AWS driver (#64). Thanks @sibowsb!

0.10.1 (2020-04-20)

Features

Add support to mimic xattr in Windows by storing data in .<filename>.xattr (#56). Thanks @RangelReale.
Add support for python 3.8.

Bugs

Freeze package azure to version 4.0.0 (#58).
Drop support for python 3.4 due to PyYAML requirement !=3.4.*.

0.10.0 (2019-08-10)

Bugs

Azure get_blob raises TypeError when Content MD5 missing in remote (#47). Thanks @matt-carr.
AzureStorageDriver reads stream twice without rewinding (#44). Thanks @matt-carr.
Update package dependencies to fix tests (#43).

Changes from 0.9.0:

Azure driver will issue a warning if Content MD5 is not populated.
Google driver tests will not run if credentials file is not present.

0.9.0 (2018-11-29)

Features

Driver authentication can be verified using DriverName.validate_credentials() (#34).

Changes from 0.8.0:

Initializing GoogleStorageDriver with an invalid credentials file will raise CredentialsError exception instead of CloudStorageError.

0.8.0 (2018-11-06)

Features

Blob and Container’s meta_data is now a case insensitive dictionary.
Add new driver for Minio Cloud Storage (#25). Install driver requirements with: pip install cloudstorage[minio].

Other

Move to src folder structure for package.

0.7.0 (2018-10-03)

Features

Cache-Control supported for Amazon, Google, Local, and Microsoft (#11).
Each driver’s package dependencies are now optional (#4).

Other

Remove rackspace package dependency rfc6266_parser.
Add flake8 linting and sphinx doc building to tox and travis.

0.6 (2018-07-24)

Copy metadata from setup.py to setup.cfg
Add rate limit timeout when calling google cloud storage backend during tests.
Catch UnicodeDecodeError when decoding local file attribute values.
Upgrade dependencies and include requirements.txt and dev-requirements.txt.

0.5 (2018-02-26)

Update rackspacesdk to 0.7.5 and fix broken API calls (#14).

0.4 (2017-08-29)

Implement Microsoft Azure Storage driver (#1).
Google upload_blob is failing for binary stream (#7 and #8).
Fixed type annotations using mypy.
Formatted code using flake8 recommendations.

0.3 (2017-05-24)

Fixes #6: Add kwargs to each driver’s init method.

0.2 (2017-04-21)

Add pip cache to travis yml file to speed up tests.
Set wheel python-tag to py3 only
Set tox to pass all env variables to py.test
Add travis repo encrypted env variables for running tests.

0.1 (2017-04-20)

First release.

Release files for cloudstorage 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for cloudstorage 0.11.0
File	Interpreter	ABI	Platform	Reset
cloudstorage-0.11.0-py3-none-any.whl 54.2 kB Jan 15, 2021	Python 3	none	any	Details

Release files / cloudstorage-0.11.0-py3-none-any.whl

Download URL	cloudstorage-0.11.0-py3-none-any.whl
Size	54.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`dcbcc4b4abe29f1af9953ebf45251ff4468ea41ae05acbc613aaca58268b65fe`
BLAKE2b-256 checksum How to use checksums	`f02575ecee40bd41e3ab4fcc9c191094f22df347f6f21779c78be9de13e32669`
Upload date	Jan 15, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.3.0 pkginfo/1.6.1 requests/2.25.1 setuptools/51.1.2 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.1`