AWS metadata as dataframes
Project description
Submodules
awsdf.aws module
This module enables connecting to AWS and extracting metadata in pandas dataframes.
Installing from PyPI: pip install -U awsdf
USAGE:
import awsdf
aws_account = awsdf.Account(profile_name=”<PROFILE_NAME>”)
glue_databases_df = aws_account.glue_get_databases()
class awsdf.aws.Account(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, profile_name=None)
Instantiate class object for connecting to AWS and retriving metadata from AWS
__init__(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, profile_name=None)
Provide access keys OR Profile name to connect to AWS account. Keys take preceedence
Parameters:
aws_access_key_id (string) – AWS access key ID
aws_secret_access_key (string) – AWS secret access key
aws_session_token (string) – AWS temporary session token
region_name (string) – AWS region
profile_name (string) – AWS profile name
glue_get_jobs() -> DataFrame
Get AWS Glue jobs
- Returns:
dataframe
glue_get_job_history(job_name, no_of_runs=1) -> DataFrame
Retrieve glue job history
- Arguments:
job_name – Name of job to retrive history
- Keyword Arguments:
no_of_runs – No of runs to retrive in descending order (default: {1})
- Returns:
dataframe
glue_get_databases() -> DataFrame
Get AWS Glue jobs
- Returns:
dataframe
glue_get_tables(dbname=None) -> DataFrame
Get AWS Glue tables
- Keyword Arguments:
dbname – Database Name for which to retrieve tables (default: {None})
- Returns:
dataframe
glue_get_fields(dbname, tablename) -> DataFrame
Get AWS Glue table columns
- Keyword Arguments:
dbname – Database Name for table tablename – Database Name for which to retrieve columns
- Returns:
dataframe
athena_data_dictionary(include_dbs: list = [], exclude_dbs: list = []) -> DataFrame
Get AWS Athean data dictionary. A data frame with all databases, tables & fields with datatypes
- Keyword Arguments:
include_dbs (optional) – list of databases to be included exclude_dbs (optional) – list of databases to be excluded if include_dbs list is empty.
- Returns:
dataframe
quicksight_get_datasources() -> DataFrame
Get QuickSight datasources
- Keyword Arguments:
N/A
- Returns:
dataframe
quicksight_get_datasets(includeDetails: bool = False) -> DataFrame
Get QuickSight datasets
- Keyword Arguments:
includeDetails (optional) – Include addition details i.e. ConsumedSpiceCapacityInBytes & Owner. Default=False
- Returns:
dataframe
quicksight_get_dataset_permissions(AwsAccountId: str, DataSetId: str)
Get QuickSight dataset permissions
- Keyword Arguments:
AwsAccountId – AWS account id DataSetId – Dataset id
- Returns:
dataframe
quicksight_get_dataset_details(datasetId: str) -> dict
Get QuickSight dataset details
- Keyword Arguments:
DataSetId – Dataset id
- Returns:
dataframe
quicksight_get_dashboards(includeDetails: bool = False) -> DataFrame
Get QuickSight dashboards
- Keyword Arguments:
includeDetails (optional) – NOT IMPLEMENTED Include addition details. Default=False
- Returns:
dataframe
quicksight_get_dashboard_details(dashboardId: str) -> dict
Get QuickSight dashboard details
- Keyword Arguments:
dashboardId – Dashboard id
- Returns:
dictionary
Module contents
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.