Why TracePipe?

Data pipelines are black boxes. Rows vanish. Values change. You're left guessing.

df = pd.read_csv("customers.csv")
df = df.dropna()                      # Some rows disappear
df = df.merge(regions, on="zip")      # New rows appear, some vanish
df["income"] = df["income"].fillna(0) # Values change silently
df = df[df["age"] >= 18]              # More rows gone
# What happened to customer C-789? 🤷

TracePipe gives you the complete audit trail — zero code changes required.

Getting Started

pip install tracepipe

import tracepipe as tp
import pandas as pd

tp.enable(mode="debug", watch=["income"])

df = pd.read_csv("customers.csv")
df = df.dropna()
df["income"] = df["income"].fillna(0)
df = df[df["age"] >= 18]

tp.check(df)  # See what happened

TracePipe Check: [OK] Pipeline healthy

Retention: 847/1000 (84.7%)
Dropped: 153 rows
  • DataFrame.dropna: 42
  • DataFrame.__getitem__[mask]: 111

Value changes: 23 cells modified
  • DataFrame.fillna: 23 (income)

That's it. One import, full visibility.

Core API

Key Features

Real-World Example

import tracepipe as tp
import pandas as pd

tp.enable(mode="debug", watch=["age", "income", "label"])

# Load and clean
df = pd.read_csv("training_data.csv")
df = df.dropna(subset=["label"])
df["income"] = df["income"].fillna(df["income"].median())
df = df[df["age"] >= 18]

# Audit
print(tp.check(df))

Retention: 8234/10000 (82.3%)
Dropped: 1766 rows
  • DataFrame.dropna: 423
  • DataFrame.__getitem__[mask]: 1343

Value changes: 892 cells
  • DataFrame.fillna: 892 (income)

# Why does this customer have a filled income?
tp.why(df, col="income", where={"customer_id": "C-789"})

Cell History: row 156, column 'income'
  Current value: 45000.0
  [i] Was null at step 1 (later recovered)

  History (1 change):
    None -> 45000.0
      by: DataFrame.fillna

Two Modes

tp.enable(mode="ci")     # Lightweight
tp.enable(mode="debug")  # Full lineage

What's Tracked

Data Quality Contracts

(tp.contract()
    .expect_unique("customer_id")
    .expect_no_nulls("email")
    .expect_retention(min_rate=0.9)
    .check(df)
    .raise_if_failed())

Documentation

📚 Full Documentation

• Quickstart
• User Guide
• API Reference
• Examples

Known Limitations

TracePipe tracks cell mutations, merge provenance, concat provenance, and duplicate drop decisions reliably. A few patterns have limited tracking:

Contributing

git clone https://github.com/gauthierpiarrette/tracepipe.git
cd tracepipe
pip install -e ".[dev]"
pytest tests/ -v

See CONTRIBUTING for guidelines.

License

MIT License. See LICENSE.

pip install tracepipe