Reusable utilities for working with Glue PySpark jobs
Project description
glue-utils
Description
Reusable utilities for working with Glue PySpark jobs
Installation
As a runtime (or production) dependency...
pip install glue-utils
For development...
This library does not include pyspark and aws-glue-libs as
dependencies as they are already pre-installed in Glue's runtime
environment.
To help in developing your Glue jobs locally in your IDE, it is helpful
to install pyspark and aws-glue-libs. Unfortunately, aws-glue-libs
is not available through PyPI so we can only install it from its git
repository.
pip install pyspark==3.3.0
pip install git+https://github.com/awslabs/aws-glue-libs.git@master
To make your local environment as close to Glue's runtime as possible, use the versions specified in this document.
Documentation
For more details on what you can use this library for, check out the project wiki.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for glue_utils-0.1.1-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ae76b9f98c9d3d76bd1616bb7636f004f13855f51264929353d136c2ff7b5db3 |
|
| MD5 | fad89e2968e2689382d10011523ddbc9 |
|
| BLAKE2b-256 | eafe3aacadd79cd0d540e8934e571fe5f168a29161a325e9fcd88bfcb50e135b |
