Timefence

Your ML model may be trained on the future. Find out in one command.

Website · Docs · Changelog · Contributing

Timefence finds and fixes temporal data leakage in ML training sets. No infrastructure required — runs locally, reads Parquet/CSV, and finishes in seconds.

If you build training data by joining features to labels, your model may be training on the future. A LEFT JOIN or merge_asof gives each label the latest feature row — including data from after the event you're predicting. The model trains on the future. Offline metrics look great. Production doesn't match. No error, no warning, no way to tell from the output alone.

pip install timefence

Try It in 60 Seconds

timefence quickstart churn-example && cd churn-example
timefence audit data/train_LEAKY.parquet

TEMPORAL AUDIT REPORT
Scanned 5,000 rows

WARNING  LEAKAGE DETECTED in 3 of 4 features

  LEAK  rolling_spend_30d
        1,520 rows (30.4%) use feature data from the future
        Severity: HIGH

  LEAK  days_since_login
        4,909 rows (98.2%) use feature data from the future
        Severity: HIGH

  OK    user_country - clean (5,000 rows)
  OK    account_age_days - clean (5,000 rows)

Rebuild it with temporal correctness:

timefence build --labels data/labels.parquet --features features.py --output train_CLEAN.parquet

Building training set...

  Labels     5,000 rows from data/labels.parquet
  Features   4 features

  Joining with point-in-time correctness (feature_time < label_time):

  OK  user_country         5,000 / 5,000 matched
  OK  account_age_days     5,000 / 5,000 matched
  OK  rolling_spend_30d    5,000 / 5,000 matched
  OK  days_since_login     5,000 / 5,000 matched

  Written   train_CLEAN.parquet (5,000 rows, 7 cols)

Verify:

timefence audit train_CLEAN.parquet
# ALL CLEAN - no temporal leakage detected

Audit Your Existing Data

You don't need to change your pipeline. Point Timefence at any training set you already have:

timefence audit your_training_set.parquet --features features.py --keys user_id --label-time label_time

If it's clean, you'll know. If it's not, you'll see exactly which features leak, how many rows, and the severity. Takes seconds.

Python API

Audit any existing dataset — no sources or feature definitions needed:

import timefence

report = timefence.audit("train.parquet", keys=["user_id"], label_time="label_time")
report.assert_clean()  # raises if leakage found

Or define sources and features to build a correct dataset from scratch:

users = timefence.Source(path="data/users.parquet", keys=["user_id"], timestamp="updated_at")
txns  = timefence.Source(path="data/txns.parquet", keys=["user_id"], timestamp="created_at")

country = timefence.Feature(source=users, columns=["country"])
spend   = timefence.Feature(source=txns, embargo="1d", name="spend_30d", sql="""
    SELECT user_id, created_at AS feature_time,
           SUM(amount) OVER (PARTITION BY user_id ORDER BY created_at
               RANGE BETWEEN INTERVAL 30 DAY PRECEDING AND CURRENT ROW) AS spend_30d
    FROM {source}
""")

labels = timefence.Labels(
    path="data/labels.parquet", keys=["user_id"],
    label_time="label_time", target=["churned"],
)

result = timefence.build(labels=labels, features=[country, spend], output="train.parquet")

Add to CI

Stop leakage before it reaches production:

- run: pip install timefence && timefence audit data/train.parquet --features features.py --strict

--strict exits with code 1 on leakage. Your pipeline fails before a leaky model ever trains.

Performance

Built on DuckDB's columnar engine. Median of 3 runs after warmup (Intel i7, 16 GB):

Adding embargo, staleness, and splits costs seconds, not minutes.

uv run python benchmarks/bench.py --quick
uv run python benchmarks/bench.py --quick --include-pandas

How It Works

Timefence generates SQL (ASOF JOIN or ROW_NUMBER) and runs it in an embedded DuckDB. No server, no JVM, no Spark. It enforces one rule — feature_time < label_time - embargo — for every row, every feature, every build. Every query is inspectable via timefence -v build or timefence explain.

All Features

What Timefence Is NOT

One tool. One job. Temporal correctness for ML training data.

Documentation · Contributing · Changelog

MIT License