Skip to main content

Simplify use of AWS resources in your code with aws-python-utils

Project description

aws-python-utils

Simplify use of AWS resources in your code with aws-python-utils

Install

pip install aws-python-util

SecretManager Util

  • Easy secret retrieval
get_secret()

Use ENVIRONMENT variable "AWS_SECRET_MANAGER_SECRET" or pass in secret_key name.

Example Usage

Via ENVIRONMENT variable

from aws_python_utils import secretmanager
import os

os.environ["AWS_SECRET_MANAGER_SECRET"] = "mysecret"
secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret()

Or pass in secret_key name ..

from aws_python_utils import secretmanager
import os

secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret(secret_key="mysecret")

S3 Util

  • Streams an s3 object directly into a pandas DataFrame to avoid writing to disk and then loading from disk
  • Uploads a DataFrame directly to s3
Example Usage
from aws_python_utils import s3
from io import BytesIO
import pandas as pd
import numpy as np

bucket,key = s3.AwsS3.get_bucket_and_key_from_s3_path("s3://my-bucket/mypath/to/object")

print("bucket = " + bucket)  # my-bucket
print("key = " + key)        # mypath/to/object

# download a tab separated file schema: id    val1  val2
df = s3.AwsS3.download_s3_file(s3_path, header=0, sep='\t', index='id')

df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)), columns=['a', 'b', 'c', 'd', 'e'])
io_buffer = BytesIO()
df2.to_csv(io_buffer, columns=['a', 'c', 'e'], sep='\t', index=False)

s3.AwsS3.upload_to_s3("s3://your-bucket/path/to/object.tsv", io_buffer)

EC2 Util

  • Ami cleaner to mop up old ami's
Example Usage
from aws_python_utils.ec2 import AwsEC2
from datetime import datetime

ec2 = AwsEC2()
# keep latest 5 ami'
ec2.clean_images("my-service-ami-dev-*", num_to_keep=5, cutoff_date=datetime(2018, 8, 1), images_to_keep=['ami-keepmeid'])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
aws_python_utils-0.0.7.tar.gz (6.0 kB) Copy SHA256 hash SHA256 Source None Aug 17, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page