Skip to main content

Simple Smart Pipe Operator

Project description

Downloads Build Status PyPI

Simple Smart Pipe

SSPipe is a python productivity-tool for rapid data manipulation in python.

It helps you break up any complicated expression into a sequence of simple transformations, increasing human-readability and decreasing the need for matching parentheses!

As an example, here is a single line code for reading students' data from 'data.csv', reporting those in the class 'A19' whose score is more than the average class score into 'report.csv':

from sspipe import p, px
import pandas as pd

pd.read_csv('data.csv') | px[px['class'] == 'A19'] | px[px.score > px.score.mean()].to_csv('report.csv')

As another example, here is a single line code for plotting sin(x) for points in range(0, 2*pi) where cos(x) is less than 0 in red color:

from sspipe import p, px
import numpy as np
import matplotlib.pyplot as plt

np.linspace(0, 2*np.pi, 100) | px[np.cos(px) < 0] | p(plt.plot, px, np.sin(px), 'r')

# The single-line code above is equivalent to the following code without SSPipe:
# X = np.linspace(0, 2*np.pi, 100)
# X = X[np.cos(X) < 0]
# plt.plot(X, np.sin(X), 'r')

If you're familiar with | operator of Unix, or %>% operator of R's magrittr, sspipe provides the same functionality in python.

Installation and Usage

Install sspipe using pip:

pip install --upgrade sspipe

Then import it in your scripts.

from sspipe import p, px

The whole functionality of this library is exposed by two objects p (as a wrapper for functions to be called on the piped object) and px (as a placeholder for piped object).

Examples

Description Python expression using p and px Equivalent python code
Simple
function call
"hello world!" | p(print) X = "hello world!"
print(X)
Function call
with extra args
"hello" | p(print, "world", end='!') X = "hello"
print(X, "world", end='!')
Explicitly positioning
piped argument
with px placeholder
"world" | p(print, "hello", px, "!") X = "world"
print("hello", X, "!")
Chaining pipes 5 | px + 2 | px ** 5 + px | p(print) X = 5
X = X + 2
X = X ** 5 + X
print(X)
Tailored behavior
for builtin map
and filter
(
range(5)
| p(filter, px % 2 == 0)
| p(map, px + 10)
| p(list) | p(print)
)
X = range(5)
X = filter((lambda x:x%2==0),X)
X = map((lambda x: x + 10), X)
X = list(X)
print(X)
NumPy expressions range(10) | np.sin(px)+1 | p(plt.plot) X = range(10)
X = np.sin(X) + 1
plt.plot(X)
Pandas support people_df | px.loc[px.age > 10, 'name'] X = people_df
X.loc[X.age > 10, 'name']
Assignment people_df['name'] |= px.str.upper() X = people_df['name']
X = X.str.upper()
people_df['name'] = X
Pipe as variable to_upper = px.strip().upper()
to_underscore = px.replace(' ', '_')
normalize = to_upper | to_underscore
" ab cde " | normalize | p(print)
_f1 = lambda x: x.strip().upper()
_f2 = lambda x: x.replace(' ','_')
_f3 = lambda x: _f2(_f1(x))
X = " ab cde "
X = _f3(X)
print(X)
Builtin
Data Structures
2 | p({px-1: p([px, p((px+1, 4))])}) X = 2
X = {X-1: [X, (X+1, 4)]}

How it works

The expression p(func, *args, **kwargs) returns a Pipe object that overloads __or__ and __ror__ operators. This object keeps func and args and kwargs until evaluation of x | <Pipe>, when Pipe.__ror__ is called by python. Then it will evaluate func(x, *args, **kwargs) and return the result.

The px object is simply p(lambda x: x).

Please notice that SSPipe does not wrap piped objects. On the other hand, it just wraps transforming functions. Therefore, when a variable like x is not an instance of Pipe class, after python evaluates y = x | p(func), the resulting variable y has absolutely no trace of Pipe. Thus, it will be exactly the same object as if we have originally evaluated y = func(x).

Common Gotchas

  • Incompatibility with dict.items, dict.keys and dict.values:

    The objects returned by dict.keys(), dict.values() and dict.items() are called view objects. Python does not allow classes to override the | operator on these types. As a workaround, the / operator has been implemented for view objects. Example:

    # WRONG ERRONEOUS CODE:
    {1: 2, 3: 4}.items() | p(list) | p(print)
    
    # CORRECT CODE (With / operator):
    {1: 2, 3: 4}.items() / p(list) | p(print)
    

Compatibility with JulienPalard/Pipe

This library is inspired by, and depends on, the intelligent and concise work of JulienPalard/Pipe. If you want a single pipe.py script or a lightweight library that implements core functionality and logic of SSPipe, Pipe is perfect.

SSPipe is focused on facilitating usage of pipes, by integration with popular libraries and introducing px concept and overriding python operators to make pipe a first-class citizen.

Every existing pipe implemented by JulienPalard/Pipe library is accessible through p.<original_name> and is compatible with SSPipe. SSPipe does not implement any specific pipe function and delegates implementation and naming of pipe functions to JulienPalard/Pipe.

For example, JulienPalard/Pipe's example for solving "Find the sum of all the even-valued terms in Fibonacci which do not exceed four million." can be re-written using sspipe:

def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

from sspipe import p, px

euler2 = (fib() | p.where(lambda x: x % 2 == 0)
                | p.take_while(lambda x: x < 4000000)
                | p.add())

You can also pass px shorthands to JulienPalard/Pipe API:

euler2 = (fib() | p.where(px % 2 == 0)
                | p.take_while(px < 4000000)
                | p.add())

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sspipe-0.1.17.tar.gz (10.6 kB view details)

Uploaded Source

Built Distribution

sspipe-0.1.17-py3-none-any.whl (8.5 kB view details)

Uploaded Python 3

File details

Details for the file sspipe-0.1.17.tar.gz.

File metadata

  • Download URL: sspipe-0.1.17.tar.gz
  • Upload date:
  • Size: 10.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.10.4

File hashes

Hashes for sspipe-0.1.17.tar.gz
Algorithm Hash digest
SHA256 21f14760901c1d45793c826f36c144112f4ada382e07a2d6d566bb8bddab7b77
MD5 3161ef36b6da07814071c8d2abc7fe37
BLAKE2b-256 d4d189b35880c8f25d01a1802322567fd831fdd5b7d62ec3649df85bf3e3e05e

See more details on using hashes here.

File details

Details for the file sspipe-0.1.17-py3-none-any.whl.

File metadata

  • Download URL: sspipe-0.1.17-py3-none-any.whl
  • Upload date:
  • Size: 8.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.10.4

File hashes

Hashes for sspipe-0.1.17-py3-none-any.whl
Algorithm Hash digest
SHA256 4d600ea418ad03d0e28e99cd0080399d54f3a85eb5815dd7fbfa982df685b91c
MD5 791f522e69bdcf7f6f3c8d5fe5662cbb
BLAKE2b-256 0ada8466d83beb78803ade04d43dc9931e3a4546bb6f975df270c3c53c8b2fd2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page