Skip to main content

No project description provided

Project description

Introduction

Emporium provides an abstraction over file stores. It currently supports storing files on local disk, in memory, or on S3. Most stores are implemented as a simple wrapper around the smart-open package.

Example

The following code snippet creates a store, writes some text to a file in the store, and then reads and prints the file twice. The file hello.txt will have been created under folder data/job, relative to the current working directory.

from emporium import create_store

config = {
    "type": "local",
    "base_path": "data"
}

store = create_store(config)

with store.write("job/hello.txt") as handle:
    handle.write("world!")

with store.read("job/hello.txt") as handle:
    print(handle.read())

substore = store.substore("job/")

with substore.open("hello.txt") as handle:
    print(handle.read())

To write the file to a key in S3, change the config definition to the following:

config = {
     "type": "s3",
     "bucket": "my-data-bucket",
     "prefix": "data"
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

emporium-0.12.0.tar.gz (7.7 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page