Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Utility package to AWS Athena and AWS Glue.

Project description

Lahap

Lahap is a utility package for AWS Athena and AWS Glue.

Code Style: Black

Usage

In order to run Lahap functions you must instantiate a Lahap session, a boto3.Session wrapper to manage boto3 calls. Use can provide any valid boto3.Session parameter to it.

from lahap import create_session

lahap = create_session(region_name="us-east-1", profile_name="profile") # using profile
lahap = create_session(region_name="us-east-1", aws_access_key_id="access-key", 
                       aws_secret_access_key="secret-key") # using explicit key credentials

Truncate table

Deletes all S3 files located in a Glue Table's S3 location. Be careful.

lahap.truncate_table(database="catalog-database", table="catalog-table")

Drop table

Drops Glue Table with or without its respective data in S3. Be careful.

lahap.drop_table(database="catalog-database", table="catalog-table", only_schema=False)

Convert table to Parquet

Copies a table storing it as Parquet files through CTA.

lahap.convert_table_to_parquet(
    query_database="source-database",
    query_table="source-table",
    compression="parquet-compression", # "UNCOMPRESSED", "SNAPPY", "LZO", "GZIP"
    result_database="result-database",
    result_table="result-table-parquet",
    external_location="s3://my-bucket/path",
)

Convert query to Parquet

Create a new table from query and storing it as Parquet files through CTA.

lahap.convert_query_to_parquet(
    query="SELECT * FROM database.table",
    compression="parquet-compression", # "UNCOMPRESSED", "SNAPPY", "LZO", "GZIP"
    result_database="result-database",
    result_table="result-table-parquet",
    external_location="s3://my-bucket/path",
)

References

Amazon Athena CTA

https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for Lahap, version 0.2
Filename, size File type Python version Upload date Hashes
Filename, size Lahap-0.2-py3-none-any.whl (5.9 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size Lahap-0.2.tar.gz (3.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page