Skip to main content

Simplify use of AWS resources in your code with aws-python-utils

Project description


Simplify use of AWS resources in your code with aws-python-utils


pip install aws-python-util

SecretManager Util

  • Easy secret retrieval

Use ENVIRONMENT variable "AWS_SECRET_MANAGER_SECRET" or pass in secret_key name.

Example Usage

Via ENVIRONMENT variable

from aws_python_utils import secretmanager
import os

os.environ["AWS_SECRET_MANAGER_SECRET"] = "mysecret"
secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret()

Or pass in secret_key name ..

from aws_python_utils import secretmanager
import os

secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret(secret_key="mysecret")

S3 Util

  • Streams an s3 object directly into a pandas DataFrame to avoid writing to disk and then loading from disk
  • Uploads a DataFrame directly to s3
Example Usage
from aws_python_utils import s3
from io import BytesIO
import pandas as pd
import numpy as np

bucket,key = s3.AwsS3.get_bucket_and_key_from_s3_path("s3://my-bucket/mypath/to/object")

print("bucket = " + bucket)  # my-bucket
print("key = " + key)        # mypath/to/object

# download a tab separated file schema: id    val1  val2
df = s3.AwsS3.download_s3_file(s3_path, header=0, sep='\t', index='id')

df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)), columns=['a', 'b', 'c', 'd', 'e'])
io_buffer = BytesIO()
df2.to_csv(io_buffer, columns=['a', 'c', 'e'], sep='\t', index=False)

s3.AwsS3.upload_to_s3("s3://your-bucket/path/to/object.tsv", io_buffer)

EC2 Util

  • Ami cleaner to mop up old ami's
Example Usage
from aws_python_utils.ec2 import AwsEC2
from datetime import datetime

ec2 = AwsEC2()
# keep latest 5 ami'
ec2.clean_images("my-service-ami-dev-*", num_to_keep=5, cutoff_date=datetime(2018, 8, 1), images_to_keep=['ami-keepmeid'])

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aws-python-utils, version 0.0.7
Filename, size File type Python version Upload date Hashes
Filename, size aws_python_utils-0.0.7.tar.gz (6.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page