plumbum
A plumbing syntax for Python that provides a clear distinction between data and operators, enabling composable function pipelines.
Overview
plumbum is a library for threading data through function calls using intuitive pipe operators. Inspired by Pipe, it offers a redesigned approach with a separation between operator construction and execution.
This is primarily a syntax library focused on making data transformations more readable and composable. It is not optimized for performance-critical applications, but rather caters to the aesthetics of @habemus-papadum -- so, your mileage may vary...
Installation
pip install habemus-papadum-plumbum
Usage
# Basics
5 > add(1) | mul(2) # 12
# Iterators
[1, 2, 3] > select(mul(3) | add(1)) | where(even) | list # [4, 10]
# Async
await (5 > async_add(1) | async_mul(2)) # 12
await ([1, 2, 3] > aiter | aselect(async_add(1)) | alist) # [2, 3, 4]
# Pipelines are reusable values
op = add(1) | multiply(2)
op2 = add(3) | multiply(12)
6 > op | op2 # 204
Mental Model
- Compose first, execute later. The
|operator chains together@pb/@apboperators (and plain callables) without running them. The pipeline becomes a first-class value that you can store, combine, or pass around. - Thread data explicitly. The
>operator injects the left-hand value into the pipeline’s first argument. That works for numbers, strings, dicts, iterables, custom classes—any Python object. - Async is contagious by design. Introducing an async operator (created with
@apb, an async iterator helper, or any awaitable) upgrades the whole pipeline sovalue > pipelinereturns a coroutine. Call it withawait. - Iterables stay lazy. Synchronous helpers like
select,where,take, orchainreturn iterators; async counterparts (aselect,awhere,aiter, …) yield async iterators. Add a materializer (list,tuple,alist) when you actually need the concrete collection.
Features
- Compatible with arbitrary data types; the library focuses on syntax and composability rather than constraining what flows through the pipes.
- Seamless sync/async mixing: pipelines promote themselves to async when an async stage is present, including adapters for awaitables returned by sync operators.
- Rich iterable toolkits for sync (
pdum.plumbum.iterops) and async (pdum.plumbum.aiterops) flows, covering batching, zipping, traversal, networking, and more.(Modelled after the operators fromPipe) - jq-inspired JSON utilities (
pdum.plumbum.jq) for parsing dotted paths, querying nested data, and performing immutable transformations—plus async counterparts.
Style Guide
- Favor
|for composition. Build operators with|so you can refactor pipelines into reusable pieces. Use>sparingly—ideally once—when you finally thread data through the pipeline. Remember that|binds more tightly than>, sovalue > a | bmeansvalue > (a | b); add parentheses only when you truly need different grouping. - Iterators remain lazy by default. Iterator pipelines usually yield another iterator. In tests or scripts, append a materializer such as
| listwhen you need concrete values. Plain callables are auto-wrapped; no need forpb(list). - Sync to async promotion is automatic. Pipelines built from
@pboperators seamlessly adopt async semantics when an@apbstage (or any awaitable) appears. Keep composing with|; the result becomes awaitable. - Await async chains. Any pipeline that includes async operators returns a coroutine. Execute it with
await (value > pipeline)(or equivalentlyawait pipeline(value)). - Collect async results with
alist/acollect. Finish async iterator pipelines with| alist(alias| acollect) when you need a list in memory. - Avoid chained
>comparisons. Python treatsx > a > bas a chained comparison, which is incompatible with plumbum operators. Preferx > a | b; only use(x > a) > bwhen you intentionally need two separate execution steps.
Tutorial Notebook
Learn the details of synchronous and asynchronous pipelines by following along step-by-step in the Tutorial, which hosts the full quick-start tour covering core operators, partial application, iterable helpers, and async variants.
Acknowledgements
- plumbum draws direct inspiration from Julien Palard’s Pipe; several iterable operators (for example
select,where, anddedup) started life there before being reworked for plumbum’s APIs. - Key differences from Pipe:
- Operators are inert values:
|builds pipelines while>triggers execution, making composition and reuse explicit. - Async is a core capability: sync and async operators coexist in the same pipeline, and iterator helpers come in both synchronous and asynchronous flavors.
- The bundled
pdum.plumbum.jqmodule adds a jq-inspired path/query layer, expanding the original operator toolbox into structured-data transformations.
- Operators are inert values:
- The codebase was authored primarily with AI assistance—roughly 95% by OpenAI Codex and 5% by Claude Code—with artistic direction and stewardship by @habemus-papadum.
Development
This project uses UV for dependency management.
Setup
# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# Clone the repository
git clone https://github.com/habemus-papadum/pdum_plumbum.git
cd pdum_plumbum
# Provision the entire toolchain (uv sync, pnpm install, widget build, pre-commit hooks)
./scripts/setup.sh
Important for Development:
./scripts/setup.shis idempotent—rerun it after pulling dependency changes- Use
uv sync --frozento ensure the lockfile is respected when installing Python deps
Running Tests
# Run all tests
uv run pytest
# Run a specific test file
uv run pytest tests/test_example.py
# Run a specific test function
uv run pytest tests/test_example.py::test_version
# Run tests with coverage
uv run pytest --cov=src/pdum/plumbum --cov-report=xml --cov-report=term
Code Quality
# Check code with ruff
uv run ruff check .
# Format code with ruff
uv run ruff format .
# Fix auto-fixable issues
uv run ruff check --fix .
Building
# Build Python
./scripts/build.sh
# Or build just the Python distribution artifacts
uv build
Releasing
Releases run from the release GitHub Actions workflow, which calls
ship:
# Plan first (the default), then release for real
gh workflow run release.yml -f bump=minor -f dry_run=true
gh workflow run release.yml -f bump=minor -f dry_run=false
Each release is tagged habemus-papadum-plumbum-v<version> on the commit it
was built from. Between releases pyproject.toml carries <last release>+dev; ship sets the release version, builds, tests, publishes to
PyPI, tags, and restores the development marker. To see the plan locally:
uv run ship release habemus-papadum-plumbum --bump minor --dry-run
Install the git hooks (including the ship-check hook) with
uv run pre-commit install; ./scripts/setup.sh does this.
Automation scripts
./scripts/setup.sh– bootstrap uv and pre-commit hooks./scripts/build.sh– sync the environment the way CI does./scripts/test_notebooks.sh– execute demo notebooks (uses./scripts/nb.shunder the hood)./scripts/setup-visual-tests.sh– install Playwright browsers for visual tests
License
MIT License - see LICENSE file for details.
Metadata
Release files for habemus-papadum-plumbum 0.6.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| habemus_papadum_plumbum-0.6.1.tar.gz | 166.6 kB | Without provenance | Details for habemus_papadum_plumbum-0.6.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| habemus_papadum_plumbum-0.6.1-py3-none-any.whl | Python 3 | none | any | Details for habemus_papadum_plumbum-0.6.1-py3-none-any.whl |
Total release size: 191.0 kB
Release files / habemus_papadum_plumbum-0.6.1.tar.gz
| Download URL | habemus_papadum_plumbum-0.6.1.tar.gz |
|---|---|
| Size | 166.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7ee6f091e2bba7a1ec6ce1a83929b5d8b1ed8c31b9550d568b9da0f1501a078c
|
|
BLAKE2b-256 checksum How to use checksums |
189d9ec2aa29524f031d403c5769803615a5567750654e60f207a74d6f2be5fb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / habemus_papadum_plumbum-0.6.1-py3-none-any.whl
| Download URL | habemus_papadum_plumbum-0.6.1-py3-none-any.whl |
|---|---|
| Size | 24.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6e5f9bf185a3d2d3d585ccdd7ad83ecf530b4f0e468ac84ea4c60fe906b1262a
|
|
BLAKE2b-256 checksum How to use checksums |
4868f110d740d7843d1bb00c3d12c4f8d34d0198d5d3c8ff86707acd1ebcf7cb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for habemus-papadum-plumbum 0.6.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.