pairwise-comparison-framework 1.0.0

A framework for characterizing and predicting pairwise comparisons using user clustering and matrix completion

Pairwise Comparison Framework

A Python framework for clustering users based on their pairwise comparison patterns and predicting missing comparisons via matrix completion.

This project was developed as part of a final thesis (TCC) at the Escola Politécnica, Universidade Federal do Rio de Janeiro (UFRJ).

Overview

The framework operates in two stages:

1. Clustering (Stage 1): Each user's pairwise comparisons are modeled as a weighted directed graph. Shortest-path-length matrices serve as structural preference signatures, a custom distance metric measures user similarity, and MDS + K-Means produces the final clusters.

2. Prediction (Stage 2): A sparse comparison matrix is built from all users' data. Matrix completion (Iterative SVD or KNN) fills in missing entries. Applying completion within each cluster improves prediction accuracy over a global model.

Installation

Requires Python 3.13+. Install with uv:

uv sync

Or with pip:

pip install -e .

Quick Start

Clustering

import pairwise_comparison_framework as pcf

clusterizer = pcf.Clusterizer()
clusterizer.load_data("data/sample/comparisons.json", "data/sample/labels.json")

# Visualize a user's preference graph
clusterizer.plot_graph("user_id", with_labels=True)

# Get item rankings via PageRank
clusterizer.get_ranking("user_id", with_labels=True)

# Cluster users
clusters = clusterizer.get_user_clusters(n_clusters=3)

# Assess cluster quality
clusterizer.get_silhouette_scores(n_clusters=3)
clusterizer.get_elbow_data(max_clusters=10)
clusterizer.plot_clusters(n_clusters=3)

Prediction

# Iterative SVD (default)
predictor = pcf.Predictor(impute_method="svd", impute_rank=1)
predictor.load_data("data/sample/comparisons.json", "data/sample/labels.json")

# Or KNN imputation
predictor = pcf.Predictor(impute_method="knn", impute_k=5)
predictor.load_data("data/sample/comparisons.json", "data/sample/labels.json")

# Predict missing comparisons
completed_matrix = predictor.predict()

# Controlled evaluation: global vs per-cluster
results = predictor.evaluate_global_vs_clusters(
    clusters, n_rounds=50, test_size=0.2
)

Data Format

Comparisons (comparisons.json)

A JSON array of user comparison records. Each comparison score answers "How much more preferred is the first item over the second?":

• 2: first item much more preferred
• 1: first item more preferred
• 0: equally preferred (tie)
• -1: second item more preferred
• -2: second item much more preferred

[
  {
    "user_id": "user_01",
    "comparisons": [
      {
        "first_item_id": "item_01",
        "second_item_id": "item_02",
        "score": 2
      }
    ]
  }
]

Labels (labels.json, optional)

A JSON array mapping item IDs to human-readable names:

[
  {
    "item_id": "item_01",
    "label": "Oranges"
  }
]

Notebooks

Notebook	Description
notebooks/clustering.ipynb	Clustering pipeline demo with sample data
notebooks/prediction.ipynb	Prediction pipeline demo with sample data
notebooks/epfl_analysis.ipynb	Full analysis on the EPFL food preference dataset (private dataset, not included)

Project Structure

pairwise-comparison-framework/
├── src/pairwise_comparison_framework/
│   ├── __init__.py          # Exports Clusterizer, Predictor
│   ├── _base.py             # Shared data loading logic
│   ├── models/              # Pydantic data models
│   ├── clusterizer/         # Stage 1: clustering pipeline
│   └── predictor/           # Stage 2: prediction pipeline
├── data/
│   └── sample/              # Small sample dataset (wine preferences)
└── notebooks/               # Jupyter notebooks

License

This project is licensed under the MIT License. See LICENSE for details.