trecrun 0.4.0

Library for working with TREC run files

TRECRun

TRECRun is a library for working with TREC run files, with an API heavily inspired by PyTerrier's pipeline operators.

API	Operator	Description
TRECRun(results)		Create a TRECRun object from a dictionary of results or a path to a run file in TREC format.
add(self, other), subtract, multiply, divide	+, -, *, /	Perform the given operation between self's document scores and other, which can be a TRECRun or a scalar.
topk(self, k)	%	Retain only the top-k documents for each qid after sorting by score.
intersect(self, other)	&	Retain only the queries and documents that appear in both self and other.
concat(self, other)		Concat the documents in other and self, with those in other appearing at the end. Their scores will be modified to accomplish this.
normalize(self, method='rr')		Normalize scores in self using RRF (rr), min-max scaling (minmax), or standardization (standard).
write_trec_run(self, outf)		Write self to outfn in TREC format.
evaluate(self, qrels, metrics)		Compute metrics for self using qrels and return a dict mapping metric names to their values for each QID. Metrics are computed by ir_measures.

Installation

pip install trecrun

The core library has no required dependencies. Optional features:

• pip install trecrun[eval] installs ir-measures, which is required by evaluate().
• pip install trecrun[all] additionally installs smart_open, which enables loading compressed and remote run files.