Conformal prediction for machine learning classifiers

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for mariya from gravatar.com mariya

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT License Copyright (c) 2024 Mariya Lysenkova Wiklander Permission is hereby granted, free of ch...)
  • Author: Mariya Lysenkova Wiklander
  • Tags machinelearning , statistics
  • Requires: Python >=3.8
  • Provides-Extra: dev
Classifiers
Report project as malware

Project description

Python package Upload Python Package

Conformist

Conformist v1.1.1 is an implementation of conformal prediction, specifically conformal risk control. It was written using Python 3.8.

BaseCoP contains utility functions common to all conformal predictors, such as splitting data into calibration and validation sets, and setting up runs. It is extended by FNRCoP that implements conformal risk control.

The ValidationRun class contains the data from a single run, which entails shuffling the data randomly, splitting it into calibration and validation datasets, calibrating the conformal predictor on the calibration data and creating prediction sets for the validation data.

The ValidationTrial class contains a list of runs and calculates statistics across these runs.

Installation

pip install conformist

Input file format

The input to Conformist is a CSV file with the following columns: id, known_class, predicted_class, [proba_columns]

The proba_columns should contain class-specific probability scores and correspond to the names used in the known_class and predicted_class columns.

Example:

id known_class predicted_class ClassA ClassB ClassC
Sample1 ClassB ClassA 0.70 0.25 0.05
Sample2 ClassA ClassA 0.98 0.02 0.0
Sample3 ClassC ClassC 0.01 0.01 0.98

Example implementation

from conformist import FNRCoP, PredictionDataset, ValidationTrial

CALIB_DATA_CSV = 'path/to/calib.csv'
TEST_DATA_CSV = 'path/to/test.csv'
OUTPUT_DIR = 'path/to/output'
ALPHA = 0.05 # Select a reasonable False Negative Rate

# Read in formatted predictions
calpd = PredictionDataset(predictions_csv=CALIB_DATA_CSV,
                          dataset_name='my_cal_data')
testpd = PredictionDataset(predictions_csv=TEST_DATA_CSV,
                           dataset_name='my_test_data')

# Get class counts and prediction heatmap
calpd.run_reports(OUTPUT_DIR)

# Validation trial and reports
mcp = FNRCoP(calpd, alpha=ALPHA)
trial = mcp.do_validation_trial(n_runs=10000)
trial.run_reports(OUTPUT_DIR)

# Recalibrate conformal predictor
mcp = FNRCoP(calpd, alpha=ALPHA)
mcp.calibrate()

# Predict
formatted_predictions = mcp.predict(testpd,
                                    OUTPUT_DIR)

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for mariya from gravatar.com mariya

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT License Copyright (c) 2024 Mariya Lysenkova Wiklander Permission is hereby granted, free of ch...)
  • Author: Mariya Lysenkova Wiklander
  • Tags machinelearning , statistics
  • Requires: Python >=3.8
  • Provides-Extra: dev
Classifiers

Release history Release notifications | RSS feed

This version

1.1.1

1.1.0

1.0.3

1.0.2

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

conformist-1.1.1.tar.gz (21.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

conformist-1.1.1-py3-none-any.whl (24.0 kB view details)

Uploaded Python 3

File details

Details for the file conformist-1.1.1.tar.gz.

File metadata

  • Download URL: conformist-1.1.1.tar.gz
  • Upload date:
  • Size: 21.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for conformist-1.1.1.tar.gz
Algorithm Hash digest
SHA256 5beb160e75e2ee94ffcdf9b58e45270ade2d34318ead7707d63af9b8835ee305
MD5 4d6a59070580195e8b584767d40015c7
BLAKE2b-256 080d10af77e551537e0ad52f6dc91c8719d9e3631c8bde85f9049e24ff0be1cb

See more details on using hashes here.

Provenance

The following attestation bundles were made for conformist-1.1.1.tar.gz:

Publisher: python-publish.yml on Molmed/conformist

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file conformist-1.1.1-py3-none-any.whl.

File metadata

  • Download URL: conformist-1.1.1-py3-none-any.whl
  • Upload date:
  • Size: 24.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for conformist-1.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 e81b2bebc8535e678527ec18823f4aa65608ce273c1406ca0ba149f94d09145d
MD5 a6a377d66357b4f3dedf5993f858cd25
BLAKE2b-256 baf1792393964674a3ab00be563f241f025433c6b93821d2d231fe8d4a2a63f0

See more details on using hashes here.

Provenance

The following attestation bundles were made for conformist-1.1.1-py3-none-any.whl:

Publisher: python-publish.yml on Molmed/conformist

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page