Skip to main content

S3 Object Store. Store and retrieve python objects in s3 simply..

Project description

S3 Object Store


Simple pythonic wrapper to use s3 as a python object store.

Provides simple object storage methods; and a Dict-like interface for more complicated uses.


Basic usage:

from s3os import store_simple, retrieve_simple, delete_simple

my_object = [1, 2, 3]

store_simple("my_key", my_object)

assert retrieve_simple("my_key") == my_object


The above example uses a global namespace in the bucket "s3os" - i.e. all the default settings of this package.

You can specify your own namespaces (i.e. buckets) as follows:

from s3os import store, retrieve, delete, ObjectLocation, Bucket

my_bucket = Bucket("my_bucket")
my_object_location = ObjectLocation("my_key", bucket=my_bucket)
my_object = [1, 2, 3]

store(my_object_location, my_object)

assert retrieve(my_object_location) == my_object


Or simply use s3 like a normal python dictionary:

from s3os import S3Dict, S3DictConfig, Bucket

my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))

# Store information in s3
s3dict["apples"] = 5
s3dict["bananas"] = 2


# Later, or in a different python executable, access the same dictionary again:
my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))

print(s3dict["apples])  # 5
print(s3dict.get_all_from_s3())  # {"apples": 5, "bananas": 2}

By default, S3Dict uses an internal cache to speed up item retrieval. Set and Delete operations are always performed synchronously.


Install the package:

pip install s3os

Setup your AWS credentials. For example set these environment variables:


Also see for more authentication options.

Note: boto.client() and Session authentication methods are not currently supported - raise an issue or submit a PR if you want them!

Development installation

Install poetry - see

The following command should be used to install the dependencies:

poetry install


The following command should be used to run the tests:

poetry run pytest tests

Valid AWS authentication credentials are required to run some of the tests. See setup instructions.

The tests make a very small number of calls to S3, so the cost of running the tests is negligible.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for s3os, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size s3os-1.0.0-py3-none-any.whl (8.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size s3os-1.0.0.tar.gz (7.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page