experimaestro-ir 0.0.0

"Experimaestro common module for IR experiments"

Information Retrieval for experimaestro

Information Retrieval module for experimaestro

The full documentation can be read at IR@experimaestro.

Install

Base experimaestro-IR can be installed with pip install xpmir. Functionalities can be added by installing optional dependencies:

• pip install xpmir[neural] to install neural-IR packages (torch, etc.)
• pip install xpmir[anserini] to install Anserini related packages

What's inside?

• Collection management (using datamaestro)
  • Interface for the IR datasets library
  • Splitting IR datasets
  • Shuffling training triplets
• Representation
  • Word Embeddings
  • HuggingFace transformers
• Indices
  • dense: FAISS interface
  • sparse: xpmir-rust library
• Standard Indexing and Retrieval
  • Anserini
• Learning to Rank
  • Pointwise
  • Pairwise
  • Distillation
  • (planned) Pipelines (e.g. ANCE)
• Neural IR
  • Cross-Encoder
  • Splade
  • DRMM
  • ColBERT
• Paper reproduction:
  • MonoBERT (Passage Re-ranking with BERT. Rodrigo Nogueira and Kyunghyun Cho. 2019)
  • (planned) DuoBERT (Multi-Stage Document Ranking with BERT. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, Jimmy Lin. 2019)
  • (planned) Splade v2 (SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval, Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. SIGIR 2021)
• Pre-trained models
  • HuggingFace integration (direct, through the Sentence Transformers library)

Examples

• BM25 retrieval
• MS Marco (cross-encoder)

Thanks

Some parts of the code have been adapted from OpenNIR