ireval 0.1.1

No project description provided

ireval

This Python package provides an implementation of the most common information retrieval (IR) metrics. Our goal is to return the same scores as trec_eval. We achieve this by extensively comparing our implementations across many different datasets with their results. ireval can be installed via

pip install ireval

Implemented metrics

The following metrics are currently implemented:

Name	Function	Description
Precision@k	precision_at_k	Precision is the fraction of retrieved documents that are relevant to the query. Precision@k considers only the documents with the highest k scores.
Precision@k%	precision_at_k_percent	Precision is the fraction of retrieved documents that are relevant to the query. Precision@k% considers only the documents with the highest k% scores.
Recall@k	recall_at_k	Recall is the fraction of the relevant documents that are successfully retrieved. Recall@k considers only the documents with the highest k scores.
Recall@k%	recall_at_k_percent	Recall is the fraction of the relevant documents that are successfully retrieved. Recall@k% considers only the documents with the highest k% scores.
Average precision	average_precision	Average precision is the area under the precision-recall curve.
R-precision	r_precision	R-Precision is the precision after R documents have been retrieved, where R is the number of relevant documents for the topic.

Usage

import ireval

relevancies = [1, 0, 1, 1, 0]
scores = [0.1, 0.4, 0.35, 0.8, .25]

p5 = ireval.precision_at_k(relevancies, scores, 5)
p5pct = ireval.precision_at_k_percent(relevancies, scores, 5)

r5 = ireval.recall_at_k(relevancies, scores, 5)
r5pct = ireval.recall_at_k_percent(relevancies, scores, 5)

ap = ireval.average_precision(relevancies, scores)
rprec = ireval.r_precision(relevancies, scores)