Auto1 ETL Challenge
Project description
Auto1 ETL Challenge
A simple ETL pipepline that serves the purpose of a challenge for AUTO1, a company that is Europe's largest wholesale platform for used cars.
Developed by Muhammad Aqib - inbox.aqib@gmail.com
Contents
auto1_etl_challenge
contains the project directory.auto1_etl_challenge\etl.py
is the main module that hasEtlPackageAuto1
class with associated methods.auto1_etl_challenge\utils.py
holds utilites that assists in performing etl operations.auto1_etl_challenge\transformations.py
module for defining custom exceptions for the required columns.auto1_etl_challenge\constants.py
holds classes for different constant values.files
is a directory for storing files for testing purpose, specifically raw, staging and transformed files.
How To Use
Package Installation
Install the package in the virtual environment
pip install auto1-etl-challenge
Code Guidelines
- Create a python file and import
EtlPackageAuto1
class from the package. - Provide source file path in the constructor.
- Call
load
andtransform
method.
from auto1_etl_challenge import EtlPackageAuto1 #import EtlPackageAuto1 class form package
EtlPackageAuto1(r'files\challenge_me.txt') #stores path of the source file
EtlPackageAuto1.load(EtlPackageAuto1.dataFile) # loads data and create staging file with cleaned data
result = EtlPackageAuto1.transform(EtlPackageAuto1.dataFile) # transform data, create transformed file and return list matrix
Output
- Staging and transformed files are created in the same directory as source file.
- List of lists is returned where first list is header while the following lists contain the values (row-wise).
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for auto1_etl_challenge-1.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89bce1d59305f3c7a35f6a0211cde0ec267aea7c8c6f11658555c4f78947e8bc |
|
MD5 | 338437c24e181fb11f987aea952ec724 |
|
BLAKE2b-256 | 322bf03cf38ba837efa3ae7655abbb865f48ec706d13718098602f85aac53a07 |
Close
Hashes for auto1_etl_challenge-1.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d97c70cc969028f97b2d047b87019e9bd8e2544b18e548f3435b53ed972d3fc8 |
|
MD5 | be9e618c40cc55a271f004563fa76949 |
|
BLAKE2b-256 | a0d8e1395460e0ad3a736a01460f7d7890dadc7b2bdc36e7dde2de2335bfa0b9 |