Skip to main content

Pivot: High-Performance Python Pipeline Tool

Change your code. Pivot knows what to run.

Pivot is a Python pipeline tool with automatic code change detection. Stages are plain Python functions; you wire them together by calling them inside a Pipeline block, and Pivot works out what needs to re-run.

  • Automatic code change detection using Python introspection
  • No paths in pipeline code: Pivot chooses where every output is stored
  • Per-stage lock files for fast parallel writes
  • Warm worker pools with preloaded imports
  • Content-addressable caching with S3 remote storage

Python: 3.13+ | Platform: Unix (Linux/macOS)

Quick Start

pip install pivot
mkdir my_project && cd my_project
git init && pivot init

Put a raw input under data/raw/:

mkdir -p data/raw
printf 'name,value\nalice,100\nbob,200\ncarol,\n' > data/raw/records.csv

Define stages and the pipeline in pipeline.py:

# pipeline.py
import pandas as pd

from pivot import Pipeline, StageParams, stage


class CleanParams(StageParams):
    min_value: int = 0


@stage
def clean(records: pd.DataFrame, params: CleanParams) -> pd.DataFrame:
    kept = records.dropna()
    return kept[kept["value"] >= params.min_value]


@stage
def summarize(rows: pd.DataFrame) -> dict:
    return {"count": len(rows), "total": int(rows["value"].sum())}


with Pipeline("demo") as pipeline:
    records = pipeline.input("records.csv")
    cleaned = clean(records, params=CleanParams(min_value=50))
    summarize(cleaned)

Then run it; Pivot resolves the DAG from the handles passed between stages:

pivot repro           # runs clean, then summarize
pivot repro           # nothing changed: both stages are cached
pivot repro --watch   # re-run automatically on file changes

Pivot writes outputs to data/demo/clean.jsonl and data/demo/summarize.yaml.

Learn More

Metadata

Release files for pivot 0.2.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pivot 0.2.2
File Size Uploaded
pivot-0.2.2.tar.gz 307.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for pivot 0.2.2
File Interpreter ABI Platform
pivot-0.2.2-py3-none-any.whl Python 3 none any Details

Total release size: 668.4 kB

Release files / pivot-0.2.2.tar.gz

Download URL pivot-0.2.2.tar.gz
Size 307.9 kB
Tags Source
SHA-256 checksum
How to use checksums
72c471df59b1a2381eb5701097cd0b3acc9f94a7c6819e12f4b6030b66c3663a
BLAKE2b-256 checksum
How to use checksums
52ff79a060bbdbf8777197a2cad537f20c5f9e239b495f26945757c7d698428b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.9.18 {"installer":{"name":"uv","version":"0.9.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / pivot-0.2.2-py3-none-any.whl

Download URL pivot-0.2.2-py3-none-any.whl
Size 360.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1e4719bac99e5f274ac32b15289479bd85e6a855585f617e5ab35c322946f2c0
BLAKE2b-256 checksum
How to use checksums
f3e4f2ffcae7dd939c795539c94cf4bbf230ab0fb8af9a801725e0cfe54781bb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.9.18 {"installer":{"name":"uv","version":"0.9.18","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release history Release notifications | RSS feed

0.3.10

2 release files

0.3.9

2 release files

0.3.8

2 release files

0.3.7

2 release files

0.3.6

2 release files

0.3.5

2 release files

0.3.4

2 release files

0.3.3

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.4

2 release files

0.2.3

2 release files

This release

0.2.2 This release

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

0.0.2

2 release files

0.0.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page