TMG ETL pipeline
Project description
TMG ETL pipeline is the base structure of our ETL pipelines. Any pipeline should inherit from TMGETLPipeline class and implement the necessary methods.
Quick Start
Installation
Install this library in a virtualenv using pip. virtualenv is a tool to create isolated Python environments. The basic problem it addresses is one of dependencies and versions, and indirectly permissions.
Supported Python Versions
Python >= 3.5
Mac/Linux
pip install virtualenv
virtualenv <your-env>
source <your-env>/bin/activate
<your-env>/bin/pip install tmg-etl-pipeline
Example Usage
from tmg_etl_pipeline.etl import TMGETLPipeline
class MyETL(TMGETLPipeline):
def run(self):
# your ETL logic goes here
# access the configs
# self.config['some-config-variable']
# access the secrets
# self.secrets['some-secret-variable']
def cleanup(self):
# your clean up code goes here
etl = MyETL(app_name='your-app-name', config_path='path/to-the-config-file')
etl.execute()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tmg-etl-pipeline-0.0.4.tar.gz
(3.3 kB
view hashes)
Built Distribution
Close
Hashes for tmg_etl_pipeline-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9472fd6d4f50aa1cda3944609e875d09c832822a8d9231ece874497be41f1e5 |
|
MD5 | 002afe6d6ab950e0dc70be2cb7838a35 |
|
BLAKE2b-256 | 1f9bd893f49053de7292fe855357ab390ea6c9def2977f7bc771e6a6392c3d12 |