S3 Object Store. Store and retrieve python objects in s3 simply..
Project description
S3 Object Store
Simple pythonic wrapper to use s3 as a python object store.
Provides simple object storage methods; and a Dict-like interface for more complicated uses.
Examples
Basic usage:
from s3os import store_simple, retrieve_simple, delete_simple
my_object = [1, 2, 3]
store_simple("my_key", my_object)
assert retrieve_simple("my_key") == my_object
delete_simple("my_key")
The above example uses a global namespace in the bucket "s3os" - i.e. all the default settings of this package.
You can specify your own namespaces (i.e. buckets) as follows:
from s3os import store, retrieve, delete, ObjectLocation, Bucket
my_bucket = Bucket("my_bucket")
my_object_location = ObjectLocation("my_key", bucket=my_bucket)
my_object = [1, 2, 3]
store(my_object_location, my_object)
assert retrieve(my_object_location) == my_object
delete(my_object_location)
Or simply use s3 like a normal python dictionary:
from s3os import S3Dict, S3DictConfig, Bucket
my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))
# Store information in s3
s3dict["apples"] = 5
s3dict["bananas"] = 2
...
# Later, or in a different python executable, access the same dictionary again:
my_bucket = Bucket("my_bucket")
s3dict = S3Dict(_config=S3DictConfig(id="my_dict_id", bucket=my_bucket))
print(s3dict["apples]) # 5
print(s3dict.get_all_from_s3()) # {"apples": 5, "bananas": 2}
By default, S3Dict
uses an internal cache to speed up item retrieval.
Set and Delete operations are always performed synchronously.
Installation
Install the package:
pip install s3os
Setup your AWS credentials. For example set these environment variables:
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
Also see https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html for more authentication options.
Note:
boto.client()
andSession
authentication methods are not currently supported - raise an issue or submit a PR if you want them!
Development installation
Install poetry - see https://pypi.org/project/poetry/
The following command should be used to install the dependencies:
poetry install
Testing
The following command should be used to run the tests:
poetry run pytest tests
Valid AWS authentication credentials are required to run some of the tests. See setup instructions.
The tests make a very small number of calls to S3, so the cost of running the tests is negligible.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file s3os-1.0.0.tar.gz
.
File metadata
- Download URL: s3os-1.0.0.tar.gz
- Upload date:
- Size: 7.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.0.0 CPython/3.8.0 Linux/5.0.0-1027-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
9970b1f9e5997b175daa59936b597cffcd4100026bfc4c670c64c041787f9a59
|
|
MD5 |
dc7c48e07da8b04435458228fa1157ce
|
|
BLAKE2b-256 |
08ee3cbcab2599caf285fc62149173069cd7d95ae7d118c9c9ae3f8aa68f4c31
|
File details
Details for the file s3os-1.0.0-py3-none-any.whl
.
File metadata
- Download URL: s3os-1.0.0-py3-none-any.whl
- Upload date:
- Size: 8.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.0.0 CPython/3.8.0 Linux/5.0.0-1027-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
2f7499fb7d1c1d6a63a2e6468d9ef5ed6ee8e0ee9520317ed101e8d5339b4693
|
|
MD5 |
4511fc18e66968ff35ce93d2cb9e9979
|
|
BLAKE2b-256 |
eb572c5bb2cf96f6f01ddd8212b8e8ff4d7c5cc47178f1375ce14ff3b34ece23
|