Improving the development of Spark applications deployed as jobs on AWS services like Glue and EMR
Project description
Table of content
What is the sparksnake library?
The sparksnake library provides an easy, fast, and efficient way to use Spark features inside analytics services on AWS. With sparksnake, it is possible to use classes, methods and functions developed in pyspark to simplify, as much as possible, the journey of building Spark applications along all the particularities found in AWS services, such as Glue and EMR, for example.
Do you want to take your job Glue or your EMR cluster to the next level? Take a look at sparksnake!
Note Now the sparksnake library has an official documentation in readthedocs! Visit the following link and check out usability technical details, practical examples and more!
Features
- 🤖 Enhanced development experience of Spark Applications to be deployed as jobs in AWS services like Glue and EMR
- 🌟 Possibility to use common Spark operations for improving ETL steps using custom classes and methods
- ⚙️ No need to think too much into the hard and complex service setup (e.g. with sparksnake you can have all elements for a Glue Job on AWS with a single line of code)
- 👁️🗨️ Application observability improvement with detailed log messages in CloudWatch
- 🛠️ Exception handling already embedded in library methods
Contacts
References
Python
Docs
- Eduardo Mendes - Live de Python 189 - MkDocs
- MkDocs
- pmdown-extensions
- GitHub - MkDocs Themes
- GitHub - Material Theme for MkDocs
- Material for MkDocs - Setup
Github
- GitHub Actions - pypa/gh-action-pypi-publish
- Medium - Major, Minor and Patch
- Medium - Automate PyPI Releases with GitHub Actions
Tests
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sparksnake-0.1.21-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ff51e6cd0ee3fa1d492e643e8f777029f0d138195e39f64b48988df90071441 |
|
MD5 | 5a532c9da389e938d3eabbad8460b81c |
|
BLAKE2b-256 | 197019bfa55513547d5afea95724855f1c0ba7be9a3f8ffb678972e0e3f51f2e |