Skip to main content

General utils for interacting with aws resources.

Project description

e2fyi-utils

PyPI version Build Status Coverage Status Documentation Status Code style: black Downloads

e2fyi-utils is an e2fyi namespaced python package with utils subpackage (i.e. e2fyi.utils) which holds a collections of useful helper classes and functions to interact with various cloud resources.

API documentation can be found at https://e2fyi-utils.readthedocs.io/en/latest/.

CHANGELOG

version description
v0.1.0a1 Initial release. Support AWS S3 bucket.
v0.1.0a1.post1 Add README to pypi.
v0.1.0a2 Fix bug: include requirements.txt into setup.py

Quickstart

pip install e2fyi-utils

e2fyi.utils.aws.S3Bucket

S3 bucket with preset prefix

from e2fyi.utils.aws import S3Bucket

# upload a dict to s3 bucket
S3Bucket("foo").upload("some_folder/some_file.json", {"foo": "bar"})

# creates a s3 bucket with std prefix rule
foo_bucket = S3Bucket("foo", get_prefix=lambda prefix: "some_folder/%s" % prefix)
foo_bucket.upload("some_file.json", {"foo": "bar"})  # some_folder/some_file.json

Uploading to S3 bucket

import logging

import pandas as pd

from e2fyi.utils.aws import S3Bucket
from pydantic import BaseModel

# s3 bucket with prefix rule
s3 = S3Bucket("foo", get_prefix=lambda prefix: "some_folder/%s" % prefix)

# check if upload is successful
result = s3.upload("some_file.txt", "hello world")
if not result.is_ok:
    logging.exception(result.exception)

# upload string as text/plain file
s3.upload("some_file.txt", "hello world")

# upload dict as application/json file
s3.upload("some_file.json", {"foo": "bar"})

# upload pandas df as text/csv and/or application/json files
df = pd.DataFrame([{"key": "a", "value": 1}, {"key": "b", "value": 2}])
s3.upload("some_file.csv", df, content_type="text/csv", index=False)  # extra kwargs can be passed to pandas.to_csv method
s3.upload("some_file.json", df, content_type="application/json", orient="records")  # extra kwargs can be passed to pandas.to_json method

# upload pydantic models as application/json file
class KeyValue(BaseModel):
    key: str
    value: int
model = KeyValue(key="a", value=1)
s3.upload("some_file.json", model)

Listing contents inside S3 buckets

from e2fyi.utils.aws import S3Bucket

# list files
S3Bucket("foo").list("some_folder/")

# list files inside "some_folder/"
foo_bucket.list()

e2fyi.utils.core.Result

import logging

from e2fyi.utils.core import Result


def load_from_file(filepath: str) -> Result[string]:
    """loads the content of a file."""
    try:
        with open(filepath, "r") as fp:
            return Result(fp.read())
    except IOError as err:
        return Result(exception=err)

data = load_from_file("some_file.json")

# print with a default value fallback
print(data.with_default("default value"))

# print data if ok, else log exception
if data.is_ok:
    print(data)
else:
    logging.exception(data.exception)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for e2fyi-utils, version 0.1.0a3
Filename, size File type Python version Upload date Hashes
Filename, size e2fyi-utils-0.1.0a3.tar.gz (14.7 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page