Skip to main content

S3 Object Store. Store and retrieve python objects in s3 simply..

Project description

S3 Object Store


Simple pythonic wrapper to use s3 as a python object store.

Provides simple object storage methods; and a Dict-like interface for more complicated uses.


Basic usage:

from s3os import store_simple, retrieve_simple, delete_simple

my_object = [1, 2, 3]

store_simple("my_key", my_object)

assert retrieve_simple("my_key") == my_object


The above example uses a global namespace in the bucket "s3os" - i.e. all the default settings of this package.

You can specify your own namespaces (i.e. buckets) as follows:

from s3os import store, retrieve, delete, ObjectLocation, Bucket

my_bucket = Bucket("my_bucket")
my_object_location = ObjectLocation("my_key", bucket=my_bucket)
my_object = [1, 2, 3]

store(my_object_location, my_object)

assert retrieve(my_object_location) == my_object


Or simply use s3 like a normal python dictionary:

from s3os import S3Dict, S3DictConfig, Bucket

my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))

# Store information in s3
s3dict["apples"] = 5
s3dict["bananas"] = 2


# Later, or in a different python executable, access the same dictionary again:
my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))

print(s3dict["apples])  # 5
print(s3dict.get_all_from_s3())  # {"apples": 5, "bananas": 2}

By default, S3Dict uses an internal cache to speed up item retrieval. Set and Delete operations are always performed synchronously.


Install the package:

pip install s3os

Setup your AWS credentials. For example set these environment variables:


Also see for more authentication options.

Note: boto.client() and Session authentication methods are not currently supported - raise an issue or submit a PR if you want them!

Development installation

Install poetry - see

The following command should be used to install the dependencies:

poetry install


The following command should be used to run the tests:

poetry run pytest tests

Valid AWS authentication credentials are required to run some of the tests. See setup instructions.

The tests make a very small number of calls to S3, so the cost of running the tests is negligible.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

s3os-1.0.0.tar.gz (7.7 kB view hashes)

Uploaded Source

Built Distribution

s3os-1.0.0-py3-none-any.whl (8.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page