Skip to main content

Powerful pipeline syntax for IPython and Jupyter

Project description

pipelang

CI Status Checked with mypy License: BSD3 Python Versions PyPI Version

pipelang is an IPython extension that brings a pipe operator |> and powerful placeholder syntax extensions to IPython and Jupyter. It is:

  • Just a library you can install from PyPI, compatible with a wide range of Python 3 versions -- no fancy installation instructions, no complicated language distribution to install
  • Intended for Jupyter notebooks, for the IPython REPL, or for any interactive Python environment built on top of IPython
  • Fully compatible with all existing Python standard and third-party libraries that you already know and love

If you're familiar with the magrittr package for R, then you'll be right at home with pipelang.

Getting Started

Run the following in IPython or Jupyter to install pipelang and load the extension:

%pip install pipelang
%load_ext pipelang

The %load_ext pipelang invocation is what enables the new pipe syntax in your current session.

Features by Example

Let's look at a few examples to give a flavor of what you can do with pipelang:

# Display a sorted version of a tuple
>>> tup = (3, 4, 1, 5, 6)
>>> tup |> sorted |> tuple
(1, 3, 4, 5, 6)

The above example showcases the |>, or "pipe", operator, which is a much-loved feature of functional programming that has become increasingly mainstream. Its primary benefit is that the flow of execution follows natural left-to-right reading / writing order of the code. Whether or not such pipeline syntax is available, it's not uncommon for programmers to execute pipelines like the above multiple times during to verify the computation at each step, particularly in interactive programming environments like Jupyter. With |>, this type of incremental verification becomes a breeze: first execute tup |> sorted, then append |> tuple to execute the full chain tup |> sorted |> tuple, each time using the last-expression rendering capabilities of the notebook or REPL to inspect and verify the result.

Placeholders

The power of the |> operator is amplified via placeholder syntax for implicit lambda construction: for pipelang, we use $ to stand in for function arguments and induce function creation:

# Sort a list in reverse order
>>> lst = [3, 4, 1, 5, 6]
>>> lst |> sorted($, reverse=True)
[6, 5, 4, 3, 1]

$ is analogous to magrittr's . placeholder. It can also be used outside of pipeline contexts:

# Sort a list in reverse order and print the result
lst = [3, 4, 1, 5, 6]
reverse_sorter = sorted($, reverse=True)

# The following are equivalent:
print(reverse_sorter(lst))
lst |> reverse_sorter |> print

Each time $ appears, it represents a new argument, so sorted($, reverse=$) represents a function with two arguments:

import random

# Sort a list in either ascending or descending order with probablility 0.5:
lst = [3, 4, 1, 5, 6]
sorter = sorted($, reverse=$)
reverse = random.random() < 0.5

# The following are equivalent:
print(sorter(lst, reverse))
lst |> sorter($, reverse) |> print

Placeholders can appear anywhere -- not just as arguments to function calls:

# Sort a list and find the position of element 4:
>>> lst = [3, 4, 1, 5, 6]
>>> lst |> sorted |> $.index(3)
1

Named Placeholders

There are situations that would benefit from referencing the same placeholder multiple times, for which pipelang permits named placeholders by prefixing $ to an identifier:

# Pair even entries from a range with their adjacent odd entry
range(6) |> list |> zip($v[::2], $v[1::2]) |> list
>>> [(0, 1), (2, 3), (4, 5)]

In the above example, we could have used any name for $v, the important thing is that the same name was used -- otherwise pipelang would have induced a function with two arguments instead of one.

Undetermined Pipelines

Similar to magrittr's behavior, if any number of placeholders appear in the first step of an pipelang pipeline, this undetermined pipeline will represent a function:

>>> second_largest_value = $ |> sorted($, reverse=True) |> $[1]
>>> [3, 8, 1, 5, 6] |> second_largest_value
6

Macros and Curry Syntax

Helper Utilities

Additional Operators

Placeholder Scope

A natural question is: how does pipelang know what part of the code should be included in the body of the function induced by placeholder use? The rules are as follows:

  1. If there is a macro or pipeline step enclosing the placeholder, the induced function body includes the "smallest" such enclosing macro or pipeline step.
  2. Otherwise, the function body expands to include the nearest "chain" of function calls, attribute accesses, and / or subscript accesses.

An example of a "chain" would be something like np.array($).T.astype(int), which induces a lambda that converts its argument to a numpy array, transposes it, and then converts the result to use int64 dtype. That is, the lambda body expands to include not just np.array($), but the entire "chain" in the expression.

To see a concrete example of where this matters, consider the following two placeholder expressions:

# The following sorters do different things!
sorter1 = sorted($, key=$[1])
sorter2 = sorted($, key=f[$[1]])

sorter1 is a function that takes two arguments: a sequence, and a list of functions, the second of which will be used to compute the sort key, which it then uses to sort the first argument. sorter2, on the other hand, is a function that takes a single argument, which is a sequence that it sorts using the second element of each value in said sequence value as sort key. In most cases, sorter2 probably gives the desired behavior.

Performance Overhead

More Examples

I developed pipelang while working on Advent of Code 2025 in parallel, and used it for most of the input processesing portions of my solutions, which you can find at https://github.com/smacke/aoc2025.

What pipelang is and is not

pipelang is not a general purpose functional programming language on top of Python. It is very much not intended for production use cases, and instead caters toward quick-and-dirty one-off / scratchpad type computations in IPython and Jupyter specifically. In short, pipelang aims to provide simple but powerful pipeline and placeholder syntax to interactive Python programming environments.

All the different pipeline operators like |>, <|, *|>, etc. essentially transpile down to an instrumented variant of the bitwise-or (|) operator, and therefore every new operator left-associates at the same level of precedence, meaning that pipeline steps run from left to right in the order that they appear. pipelang aims to optimize for simplicity, readability / writability, and predictability over feature completeness (though I'd like to think it strikes a fairly good balance in this regard).

How it works

Inspiration

pipelang draws inspiration largely from magrittr, but also from efforts like coconut (a functional superset of Python), as well as from libraries like Pipe which take a different approach to fill Python's pipe gap with operator overloading hacks.

License

Code in this project licensed under the BSD-3-Clause License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pipelang-0.0.1.tar.gz (45.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pipelang-0.0.1-py2.py3-none-any.whl (26.2 kB view details)

Uploaded Python 2Python 3

File details

Details for the file pipelang-0.0.1.tar.gz.

File metadata

  • Download URL: pipelang-0.0.1.tar.gz
  • Upload date:
  • Size: 45.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.6

File hashes

Hashes for pipelang-0.0.1.tar.gz
Algorithm Hash digest
SHA256 6fee253cf5de36299e56f8f052f450310bcc6fb94a33bf218a348fe750bcb8d8
MD5 c0d5728c3d20683c221f51f37d35f301
BLAKE2b-256 11bf62910fd5aa7f1c9bfca7703d926de75c6db81d613e3e03df20d40db27e40

See more details on using hashes here.

File details

Details for the file pipelang-0.0.1-py2.py3-none-any.whl.

File metadata

  • Download URL: pipelang-0.0.1-py2.py3-none-any.whl
  • Upload date:
  • Size: 26.2 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.6

File hashes

Hashes for pipelang-0.0.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 802d48ff582ac81b1ec7c9b9e015a3206096a958c47a404be05fc19fc7bcecf4
MD5 d8a82612c7bfa06221dd80acc4deee06
BLAKE2b-256 c4c15a382d7927fbc3475691002ef68f8d1a87ba227f76cf33e3fbc33938ed91

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page