alyeska 0.3.0a1

Sample Usage

Assume an AWS IAM user with multi-factor authorization

Alyeska’s authmfa command line utility is useful for quickly assuming an AWS IAM user with MFA.

$ authmfa MyAwsUser
export AWS_ACCESS_KEY_ID=ABCDEFGHIJKLMNOPQRSTUVWXYZ
export AWS_SECRET_ACCESS_KEY=abcdefg1234567!@#$%^&
export AWS_SESSION_TOKEN=notarealsessiontoken///////5AVHwuGc*hYLp%$vr51*XTEHJjRD2JxavaD8wlJqi!aCZVhvp7nzt!U5elvoPZ@GlG%a9sT^HBrgKzQ8xZrpAADp65RYQzqvawF
$ eval `authmfa MyAwsUser`  # export to environment

Learn more about how to config this utility with authmfa -h.

Load a pandas dataframe into AWS Redshift

Large tables can be frustrating to load into Redshift. Alyeska reduces the process to a short one-line statement:

>>> aly.redpandas.insert_pandas_into(cnxn, target_table, payload_df)

In practice, it may function as

>>> import alyeska as aly
>>> import alyeska.locksmith.redshift as rs
>>> import pandas as pd

>>> cnxn = rs.connect_with_environment("my-user")
>>> target_table = "db.natural_numbers"

>>> sql = f"CREATE TABLE {target_table}(n INT NOT NULL)"
>>> aly.sqlagent.execute_sql(cnxn, sql)  # create table

>>> natural_numbers_df = pd.DataFrame({"n": range(1, 1_000_001)})
>>> aly.redpandas.insert_pandas_into(cnxn, target_table, natural_numbers_df)

Components

Tools are broken out into modules with niche purposes:

1. compose is a workflow dependency management tool

2. locksmith helps authorize AWS sessions and Redshift connections

3. logging is another thin module that standardizes logging practices

4. redpandas supports less verbose pandas/redshift functionality

5. sqlagent supports SQL executation and runtime configuration

License

This project is licensed under the Apache v2.0 License - see the LICENSE file for details.

Contribute

Begin by reading our Code of Conduct.

There are some devtools required to contribute to the repo. Create a development environment and install pre-commit to run hooks on your code.

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -r requirements.dev.txt
$ pre-commit install
$ pre-commit autoupdate

Namesake

The Alyeska Pipeline Service company maintains the Alaska pipeline; a 1200 km long pipeline connecting the oil-rich, subterranean earth in Alaska to port on the north pacific ocean.