Piper

Piper is a python package to help simplify data wrangling tasks with pandas. It provides a set of wrapper functions or 'verbs' that provide a simpler interface to standard Pandas functions.

Piper functions accept and receive pandas dataframe objects. They can be used as standalone functions but are more powerful when used together in a Jupyter notebook cell to form a data pipeline.

Instead of the traditional the 'dot notation' or method calling technique from within an object, piper receives and passes on the dataframe object between functions using the piper magic command link operator (that is '>>') within a cell. So, in traditional pandas, to see the first 5 rows of a dataframe:

df.head()

The equivalent in piper would be:

%%piper
df >> head()

Installation

To install the package, enter the following:

pip install dpiper

Documentation

Piper API documentation available at readthedocs

Quick start

Example #1

A dataframe consisting of two columns A and B.

import pandas as pd
import numpy as np

np.random.seed(42)
df = pd.DataFrame({'A': np.random.randint(10, 1000, 10),
                   'B': np.random.randint(10, 1000, 10)})
df.head()

Piper equivalent

from piper.defaults import *

%%piper
df >> assign(C = lambda x: x.A + x.B,
             D = lambda x: x.C < 1000)
   >> where("~D")

Example #2

Suppose you need the following function to trim columnar text data.

def trim_columns(df):
    ''' Trim blanks for given dataframe '''

    str_cols = df.select_dtypes(include='object').columns

    for col in str_cols:
        df[col] = df[col].str.strip()

    return df

import pandas as pd
from piper.factory import sample_data

df = sample_data()

# Select all columns EXCEPT 'dates'
subset_cols = ['order_dates', 'regions', 'countries', 'values_1', 'values_2']

criteria1 = ~df['countries'].isin(['Italy', 'Portugal'])
criteria2 = df['values_1'] > 40
criteria3 = df['values_2'] < 25

df2 = (df[subset_cols][criteria1 & criteria2 & criteria3]
       .pipe(trim_columns)
       .sort_values('countries', ascending=False))
df2.head()

Piper equivalent

Using the %%piper magic function, piper verbs can be combined with standard python functions.

from piper.defaults import *

%%piper
sample_data()
>> trim_columns()
>> select('-dates')
>> where(""" ~countries.isin(['Italy', 'Portugal']) &
              values_1 > 40 &
              values_2 < 25 """)
>> order_by('-countries')
>> head(5)

Result: