firs 0.0.8

firs: a Framework for Information Retrieval Systems

Firs: a (python) Framework for Information Retrieval Systems.

Firs is a python package, based on pyterrier, developed to help experimentation in Information Retireval.

Firs have multiple functions:

• It allows to import and evalutate traditional TREC collections
• It allows to compute an experimental Grid of Points (GoP)
• It allows to compute and handle replicates (such as shards or reformulations)

Install

To install firs, use the pip command:

pip install firs

The configuration file

To work, firs relies on a configuration file. The configuration file needs a section for the paths and a section for each of the collections that you want to work on. In the "path" section, it is mandatory to specify the path to the jdk. Notice that, firs is based on pyterrier and therefore requires a jdk ≥ 11.

an example of configuration file:

[paths]
JAVAHOME = /usr/lib/jdk-11.0.11

[collections.robust04]
runs_path = ./data/TREC/TREC_13_2004_Robust/runs/
qrel_path = ./data/TREC/TREC_13_2004_Robust/pool/qrels.robust2004.txt
coll_path = ./EXPERIMENTAL_COLLECTIONS/TIPSTER/CORPUS
shrd_path = ./data/shardings/

[collections.trec08]
runs_path = ./data/TREC/TREC_08_1999_AdHoc/runs/all/
qrel_path = ./data/TREC/TREC_08_1999_AdHoc/pool/qrels.trec8.adhoc.txt
shrd_path = ./data/shardings/

Non-public elements, such as the qrels, are not provided by firs. They need to be placed in the path specified in the configuration file. In any cases, firs can used to build runs and grid of points starting from a collection.

Initializing firs

Once the configuration file is ready, it is possible to start working with firs.

Import firs and configure it:

import firs
firs.configure(<path to configuration file>)

Iporting a collection

To import a trec collection, run

#import the metadata of the collection
collection = firs.TrecCollection(collectionName=<name of the collection>)

#import the collection: the operation might be very time consuming
collection = collection.import_collection()

The function import_collection takes nthreads as additional parameter to import the runs in a parallel fashon. If you want to import the runs using 10 processors, do:

collection = collection.import_collection(nthreads=10)

To compute the measures on the selected collection, using the given qrels, run:

measures = collection.evaluate()

Replicates

Shardings

Reformulations