Skip to main content

A Cubicweb Storage that stores the data on S3

Project description

Summary

A Cubicweb Storage that stores the data on S3.

It is a standard Cubicweb Storage, similar to the BytesFileSystemStorage. For example, if you want to store File.data payloads in S3, you should create an S3Storage instance, typically in a startup hook, as:

from cubicweb.server.hook import Hook
from cubicweb_s3storage.storages import S3Storage

class S3StorageStartupHook(Hook):
    __regid__ = 'mycube.server-startup-hook'
    events = ('server_startup', 'server_maintenance')

    def __call__(self):
        storage = S3Storage('s3_bucket')
        self.repo.system_source.set_storage('File', 'data', storage)

By default, S3 object keys (ie. S3 object identifiers) are random UUID generated at entity creation time. If you want to implement another key generation algorithm, you must write overload S3Storage.new_s3_key() method.

Communications with AWS are handled by boto3.

Configuration

The S3Storage class constructor needs an S3 bucket name.

If you want to put your aws credentials in an other file than the default (~/.aws/credentials), you must set the AWS_CONFIG_FILE environment variable.

If you want to use another profile name than default, set the AWS_PROFILE environment variable.

If you prefer to configure AWS credentials by environment variables (eg. when using docker), you can use the following variables:

  • AWS_ACCESS_KEY_ID: the access key for your AWS account,

  • AWS_SECRET_ACCESS_KEY: the secret key for your AWS account.

Please read boto3 documentation for more details.

Further configuration

In some cases you might want to use compatible S3 storage which is not on AWS.

For this you have the following variables :

  • AWS_S3_ENDPOINT_URL: endpoint url for your alternative S3 storage

This enables the use of a minio server, see their documentation about endpoints.

Note : this type of variable has been requested upstream (and related aws-cli).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cubicweb_s3storage-4.0.2.tar.gz (12.8 kB view details)

Uploaded Source

Built Distribution

cubicweb_s3storage-4.0.2-py3-none-any.whl (10.6 kB view details)

Uploaded Python 3

File details

Details for the file cubicweb_s3storage-4.0.2.tar.gz.

File metadata

  • Download URL: cubicweb_s3storage-4.0.2.tar.gz
  • Upload date:
  • Size: 12.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.3

File hashes

Hashes for cubicweb_s3storage-4.0.2.tar.gz
Algorithm Hash digest
SHA256 a58cfcf8f9032dd0667a6047870c76cb5389db50c06456a12f67d92d74d537e8
MD5 740f079c81d412bd04f58c213c18f295
BLAKE2b-256 49c7d4aed7761755d224018ba837ab0c3f4470d0e5e791aa6041e23216617707

See more details on using hashes here.

File details

Details for the file cubicweb_s3storage-4.0.2-py3-none-any.whl.

File metadata

File hashes

Hashes for cubicweb_s3storage-4.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 371bf6bf9ec3ea75f9c56fb0a7f2d28ee98c4cd379f0f3e49da62ad0629c2219
MD5 b6048595fa25c5cf5542dd567637df34
BLAKE2b-256 fb213da4dd181d3e0e0ee94e02ff3ed328a1efcc81c07f85dc7564e8d4ce24f1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page