🎯 PIT-CP

pitcp is a Python package for conformal prediction using probability integral transform (PIT) pivotal scores. Given any black-box nonconformity score, it fits a conditional density estimator on the score distribution and maps raw scores to PIT values, yielding valid marginal coverage at any user-specified level.

✨ Features

  • PIT Conformal Prediction: Maps base nonconformity scores through a learned conditional CDF, producing asymptotically exact conditional coverage.
  • Model-agnostic: Works with any callable nonconformity score s(x, y), including distance, residual, or likelihood-based scores.
  • Flexible Density Estimation: Supports normalizing flows and mixture density networks from the zuko library.
  • Marginal Coverage Guarantee: Provably valid conformal coverage at any target level via finite-sample calibration.
  • scikit-learn: Native BaseEstimator integration with a familiar fit / conformalize / predict API.

🚀 Installation

pip install pitcp

🔧 Usage

Example

import torch
import zuko
from pitcp import PITCP


def std(x):
    return torch.where((x > -0.9) & (x < 0.9), torch.cos(torch.pi * x / 2), 1.0)


def gen_data(n):
    x = torch.rand(n, 1) * 2 - 1
    return x, torch.randn(n, 1) * std(x)


torch.manual_seed(42)

(X_train, y_train), (X_cal, y_cal), (X_test, y_test) = [
    gen_data(5000) for _ in range(3)
]


# Define a nonconformity score
def score(x, y):
    return y.abs()


# Build a normalizing flow density estimator
model = zuko.flows.NSF(features=1, context=1, bins=4, hidden_features=(32, 32, 32))
optimizer = torch.optim.Adam(model.parameters(), lr=1e-2)

# Compute nonconformity scores
s_train = score(X_train, y_train)
s_cal = score(X_cal, y_cal)
s_test = score(X_test, y_test)

# Fit and conformalize
pitcp = PITCP(model, optimizer, n_epochs=10, batch_size=128)
pitcp.fit(X_train, s_train)
pitcp.conformalize(X_cal, s_cal)

# Predict conformal regions (max score thresholds) at multiple quantiles
limits = pitcp.predict(X_test, quantile=[0.7, 0.8, 0.9])

# Predict conformal coverage
covered = pitcp.predict_coverage(X_test, s_test, quantile=[0.7, 0.8, 0.9])
print(f"Empirical coverages: {covered.mean(axis=0)}")

📖 Learn More

For tutorials, API reference, visit the official site:
👉 pitcp's documentation

📊 Reproducing Results

To reproduce the experiments and figures presented in the paper, navigate to the scripts/ directory and run the corresponding Python scripts. Note that the scripts expect to be run from within the scripts/ directory to correctly access data and output paths.

cd scripts
python <script_name>.py

Script Descriptions

Script Description
convergence-plots.py Evaluates the convergence of the PIT-CP procedure across different training sample sizes using various density estimators (SOSPF, GMM).
kolmogorov-illustration.py Visualizes the Kolmogorov-Smirnov distance between conditional and marginal score distributions, illustrating the challenge PIT-CP addresses.
optimal-transport-illustration.py Illustrates the transformation of conditional score distributions into a uniform target via the Probability Integral Transform (PIT).
real-data-diagnostics.py Benchmarks PIT-CP against other conformal prediction methods (SCP, CQR, HPD, CONTRA) on real-world datasets, calculating coverage gaps and prediction interval volumes.
synthetic-plots.py Compares the conformal regions and conditional coverage of PIT-CP, CQR, and SCP on synthetic heteroscedastic data.

Figures and results will be saved in the figures/ directory.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.12.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for felixlaplante0 from gravatar.com felixlaplante0

Credits

Author: Félix Laplante

Requires

Python >=3.11

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pitcp-0.10.1.tar.gz (18.9 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pitcp-0.10.1-py3-none-any.whl (21.0 kB view details)

Uploaded Python 3

File details

Details for the file pitcp-0.10.1.tar.gz.

File metadata

  • Download URL: pitcp-0.10.1.tar.gz
  • Upload date:
  • Size: 18.9 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.14

File hashes

Hashes for pitcp-0.10.1.tar.gz
Algorithm Hash digest
SHA256 b47e629ac1f8875fb2ff8f67a6501f44796d336e8e1300d7439701932a0b80c0
MD5 f325bfccfd42f7f2204c6c4028f79e67
BLAKE2b-256 1e5bbe553270757cdd2b7728f504dabd096d1f558a432090eb84fc07fbe64d1b

See more details on using hashes here.

File details

Details for the file pitcp-0.10.1-py3-none-any.whl.

File metadata

  • Download URL: pitcp-0.10.1-py3-none-any.whl
  • Upload date:
  • Size: 21.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.14

File hashes

Hashes for pitcp-0.10.1-py3-none-any.whl
Algorithm Hash digest
SHA256 5712fc6841b09c004b7c12bd93c858c4aede6f93f462056ea6691cd7ad089a39
MD5 f7966bb102db3379d1deb4edbb030a9a
BLAKE2b-256 0ac1a5355dac7ae944e5ba209d5922f9a678679f2954c96b09e15f730efda23b

See more details on using hashes here.

Release history Release notifications | RSS feed

0.12.0

2 files

0.11.4

2 files

0.11.3

2 files

0.11.2

2 files

0.11.1

2 files

0.11.0

2 files

0.10.5

2 files

0.10.4

2 files

0.10.3

2 files

0.10.2

2 files

This release

0.10.1 This release

2 files

0.10.0

2 files

0.9.0

2 files

0.7.2

2 files

0.7.1

2 files

0.7.0

2 files

0.6.0

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.1

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page