Skip to main content

Simplify use of AWS resources in your code with aws-python-utils

Project description


Simplify use of AWS resources in your code with aws-python-utils


pip install aws-python-util



Use ENVIRONMENT variable "AWS_SECRET_MANAGER_SECRET" or pass in secret_key name.

Example Usage

Via ENVIRONMENT variable

In [1]: import os
In [2]: import secretmanager
In [3]: os.environ["AWS_SECRET_MANAGER_SECRET"] = "mysecret"
In [4]: secretmanager = secretmanager.AwsSecretManager()
In [5]: secretmanager.get_secret()
Out[5]: 'supersecretpass'

Or pass in secret_key name ..

In [1]: import os
In [2]: import secretmanager
In [3]: secretmanager = secretmanager.AwsSecretManager()
In [4]: secretmanager.get_secret(secret_key="mysecret")
Out[4]: 'supersecretpass'

S3 Util

  • Streams an s3 object directly into a pandas DataFrame to avoid writing to disk and then loading from disk
  • Uploads a DataFrame directly to s3
Example Usage
from aws_python_utils import s3
from io import BytesIO
import pandas as pd
import numpy as np

bucket,key = s3.get_bucket_and_key_from_s3_path("s3://my-bucket/mypath/to/object")

print("bucket = " + bucket)  # my-bucket
print("key = " + key)        # mypath/to/object

# download a tab separated file schema: id    val1  val2
df = s3.download_s3_file(s3_path, header=0, sep='\t', index='id')

df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)), columns=['a', 'b', 'c', 'd', 'e'])
io_buffer = BytesIO()
df2.to_csv(io_buffer, columns=['a', 'c', 'e'], sep='\t', index=False)

s3.upload_to_s3("s3://your-bucket/path/to/object.tsv", io_buffer)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aws-python-utils, version 0.0.4
Filename, size File type Python version Upload date Hashes
Filename, size aws_python_utils-0.0.4.tar.gz (5.2 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page