Squishy

Welcome to the project! Squishy is a Python package for data transformation using the Monad pattern and cleansing operations.

Installation

From PyPI:

pip install monadsquishy

From github

pip install git+https://github.com/wasit7/monadsquishy.git

Simple Transformation

Please check tutorial notebook here

load input into a dataframe

from monadsquishy import Squishy, sf
import pandas as pd

df = pd.read_parquet('./simple.parquet')

Setup transformation cofig

sq_config = {
    'transformations':[
        {
            'input_table': df,
            'transformed_path':'./staging/test1',
            'exploded_path':'./staging/test1',
            'out_columns': {
                 'country': {
                     'input':'country',
                     'funcs':[sf.country1,sf.country2,sf.country3]
                 },
                 'name': {
                     'input':'name',
                     'funcs':[lambda x:x, ],
                 },
                 'order_date': {
                     'input':'order_date',
                     'funcs':[sf.date1, sf.date2]
                 },
                 'quantity': {
                     'input':'quantity',
                     'funcs':[sf.quantity1, ]
                 },
                 'price_number': {
                     'input':'price',
                     'funcs':[sf.price1, ]
                 },
                 'price_currency': {
                     'input':'price',
                     'funcs':[sf.currency1, ]
                 } 
             }
        }
    ]
}

Run transformations

sq=Squishy(sq_config)
sq.run()

Show input dataframe

sq.input()

Show output dataframe

sq.output()

Show log dataframe

sq.log()