cloudstorage 0.11.0

Unified cloud storage API for storage services.

Cloud Storage is a Python +3.5 package which creates a unified API for the cloud storage services: Amazon Simple Storage Service (S3), Microsoft Azure Storage, Minio Cloud Storage, Rackspace Cloud Files, Google Cloud Storage, and the Local File System.

Cloud Storage is inspired by Apache Libcloud. Advantages to Apache Libcloud Storage are:

• Full Python 3 support.

• Generate temporary signed URLs for downloading and uploading files.

• Support for request and response headers like Content-Disposition.

• Pythonic! Iterate through all blobs in containers and all containers in storage using respective objects.

Usage

>>> from cloudstorage.drivers.amazon import S3Driver
>>> storage = S3Driver(key='<my-aws-access-key-id>', secret='<my-aws-secret-access-key>')

>>> container = storage.create_container('avatars')
>>> container.cdn_url
'https://avatars.s3.amazonaws.com/'

>>> avatar_blob = container.upload_blob('/path/my-avatar.png')
>>> avatar_blob.cdn_url
'https://s3.amazonaws.com/avatars/my-avatar.png'

>>> avatar_blob.generate_download_url(expires=3600)
'https://avatars.s3.amazonaws.com/my-avatar.png?'
'AWSAccessKeyId=<my-aws-access-key-id>'
'&Signature=<generated-signature>'
'&Expires=1491849102'

>>> container.generate_upload_url('user-1-avatar.png', expires=3600)
{
    'url': 'https://avatars.s3.amazonaws.com/',
    'fields': {
        'key': 'user-1-avatar.png',
        'AWSAccessKeyId': '<my-aws-access-key-id>',
        'policy': '<generated-policy>',
        'signature': '<generated-signature>'
    }
}

Supported Services

• Amazon S3

• Google Cloud Storage

• Microsoft Azure Storage

• Minio Cloud Storage

• Rackspace CloudFiles

• Local File System

Installation

To install Cloud Storage:

pip install cloudstorage

Also install the storage driver(s) you will be using:

pip install cloudstorage[amazon]
pip install cloudstorage[google]
pip install cloudstorage[local]
pip install cloudstorage[microsoft]
pip install cloudstorage[minio]
pip install cloudstorage[rackspace]

Changelog

1.0.0 (Unreleased)

Features

• Start following (Semantic Versioning).

0.11.0 (2021-01-15)

Features

• Atomic local file write (#68). Thanks @habibutsu.

• New DigitalOcean Spaces Driver (#67). Thanks @RangelReale.

Bugs

• Convert bytes to str before saving the json file for Windows Xattr simulator (#66). Thanks @RangelReale!

• Handle pathlib.Path properly for Blob.download download and Container.upload_blob (#65). Thanks @sibowsb!

• Fix Bob NotFoundError message for AWS driver (#64). Thanks @sibowsb!

0.10.1 (2020-04-20)

Features

• Add support to mimic xattr in Windows by storing data in .<filename>.xattr (#56). Thanks @RangelReale.

• Add support for python 3.8.

Bugs

• Freeze package azure to version 4.0.0 (#58).

• Drop support for python 3.4 due to PyYAML requirement !=3.4.*.

0.10.0 (2019-08-10)

Bugs

• Azure get_blob raises TypeError when Content MD5 missing in remote (#47). Thanks @matt-carr.

• AzureStorageDriver reads stream twice without rewinding (#44). Thanks @matt-carr.

• Update package dependencies to fix tests (#43).

Changes from 0.9.0:

• Azure driver will issue a warning if Content MD5 is not populated.

• Google driver tests will not run if credentials file is not present.

0.9.0 (2018-11-29)

Features

• Driver authentication can be verified using DriverName.validate_credentials() (#34).

Changes from 0.8.0:

• Initializing GoogleStorageDriver with an invalid credentials file will raise CredentialsError exception instead of CloudStorageError.

0.8.0 (2018-11-06)

Features

• Blob and Container’s meta_data is now a case insensitive dictionary.

• Add new driver for Minio Cloud Storage (#25). Install driver requirements with: pip install cloudstorage[minio].

Other

• Move to src folder structure for package.

0.7.0 (2018-10-03)

Features

• Cache-Control supported for Amazon, Google, Local, and Microsoft (#11).

• Each driver’s package dependencies are now optional (#4).

Other

• Remove rackspace package dependency rfc6266_parser.

• Add flake8 linting and sphinx doc building to tox and travis.

0.6 (2018-07-24)

• Copy metadata from setup.py to setup.cfg

• Add rate limit timeout when calling google cloud storage backend during tests.

• Catch UnicodeDecodeError when decoding local file attribute values.

• Upgrade dependencies and include requirements.txt and dev-requirements.txt.

0.5 (2018-02-26)

• Update rackspacesdk to 0.7.5 and fix broken API calls (#14).

0.4 (2017-08-29)

• Implement Microsoft Azure Storage driver (#1).

• Google upload_blob is failing for binary stream (#7 and #8).

• Fixed type annotations using mypy.

• Formatted code using flake8 recommendations.

0.3 (2017-05-24)

• Fixes #6: Add kwargs to each driver’s init method.

0.2 (2017-04-21)

• Add pip cache to travis yml file to speed up tests.

• Set wheel python-tag to py3 only

• Set tox to pass all env variables to py.test

• Add travis repo encrypted env variables for running tests.

0.1 (2017-04-20)

• First release.