Persisty Data

This is an addition to the persisty framework for handling binary data (File Uploads). The objective is to be compatible with third party services, including ones that use signed URLs for posting / reading (Such as S3).

Concepts

• DataItems represent the storage of binary data, with methods for reading and writing as well as obtaining meta such as file size, last modified and etag
• DataStores store data items, with higher level methods for copying efficiently
• DataStoreFactories handle authorization when creating create data stores, as well as providing methods for generating presigned urls meaning that the server may be bypassed completely in some implementations (such as S3)

Implementations

Chunk

The chunk implementation breaks up an item into chunks smaller than preset size and storing these using regular persisty.

Directory

The Directory Store stores data items as files within a directory on a hosted file system.

S3

The S3 Store stores data items as entries in an S3 bucket. It is unique in the stock implementations in that it also provides an S3 Data Store Factory rather than relying on the Hosted Data Store Factory used by other implementations for providing pre-signed read urls and Upload Forms.

• Reading Data
• Writing data
• Strategy for bypassing server using Signed URLs
• Example

Example

A variant of the persisty messenger example application demonstrates file upload.

Installing local development dependencies

python setup.py install easy_install "persisty-data[dev]"

Release Procedure

The typical process here is:

• Create a PR with changes. Merge these to main (The Quality workflows make sure that your PR meets the styling, linting, and code coverage standards).
• New releases created in github are automatically uploaded to pypi