Skip to main content

Utility package to AWS Athena and AWS Glue.

Project description

Lahap

Lahap is a utility package for AWS Athena and AWS Glue.

Code Style: Black

Usage

In order to run Lahap functions you must instantiate a Lahap session, a boto3.Session wrapper to manage boto3 calls. Use can provide any valid boto3.Session parameter to it.

from lahap import create_session

lahap = create_session(region_name="us-east-1", profile_name="profile") # using profile
lahap = create_session(region_name="us-east-1", aws_access_key_id="access-key", 
                       aws_secret_access_key="secret-key") # using explicit key credentials

Truncate table

Deletes all S3 files located in a Glue Table's S3 location. Be careful.

lahap.truncate_table(database="catalog-database", table="catalog-table")

Drop table

Drops Glue Table with or without its respective data in S3. Be careful.

lahap.drop_table(database="catalog-database", table="catalog-table", only_schema=False)

Convert table to Parquet

Copies a table storing it as Parquet files through CTA.

lahap.convert_table_to_parquet(
    query_database="source-database",
    query_table="source-table",
    compression="parquet-compression", # "UNCOMPRESSED", "SNAPPY", "LZO", "GZIP"
    result_database="result-database",
    result_table="result-table-parquet",
    external_location="s3://my-bucket/path",
)

Convert query to Parquet

Create a new table from query and storing it as Parquet files through CTA.

lahap.convert_query_to_parquet(
    query="SELECT * FROM database.table",
    compression="parquet-compression", # "UNCOMPRESSED", "SNAPPY", "LZO", "GZIP"
    result_database="result-database",
    result_table="result-table-parquet",
    external_location="s3://my-bucket/path",
)

References

Amazon Athena CTA

https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for Lahap, version 0.2.1
Filename, size File type Python version Upload date Hashes
Filename, size Lahap-0.2.1-py3-none-any.whl (6.0 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size Lahap-0.2.1.tar.gz (3.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page