Experimaestro common module for IR experiments
Project description
Information Retrieval for experimaestro
Information Retrieval module for experimaestro
The full documentation can be read at IR@experimaestro.
Finally, you can find the roadmap.
Install
Base experimaestro-IR can be installed with pip install xpmir
.
Functionalities can be added by installing optional dependencies:
pip install xpmir[neural]
to install neural-IR packages (torch, etc.)pip install xpmir[anserini]
to install Anserini related packages
For the development version, you can:
- If you just want the development version: install with
pip install git+https://github.com/experimaestro/experimaestro-ir.git
- If you want to edit the code: clone and then do a
pip install -e .
within the directory
What's inside?
- Collection management (using datamaestro)
- Interface for the IR datasets library
- Splitting IR datasets
- Shuffling training triplets
- Representation
- Word Embeddings
- HuggingFace transformers
- Indices
- dense: FAISS interface
- sparse: xpmir-rust library
- Standard Indexing and Retrieval
- Anserini
- Learning to Rank
- Pointwise
- Pairwise
- Distillation
- Neural IR
- Cross-Encoder
- Splade
- DRMM
- ColBERT
- Paper reproduction:
- MonoBERT (Passage Re-ranking with BERT. Rodrigo Nogueira and Kyunghyun Cho. 2019)
- (alpha) DuoBERT (Multi-Stage Document Ranking with BERT. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, Jimmy Lin. 2019)
- (beta) Splade v2 (SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval, Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. SIGIR 2021)
- (planned) ANCE
- Pre-trained models
- HuggingFace integration (direct, through the Sentence Transformers library)
Thanks
Some parts of the code have been adapted from OpenNIR
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
experimaestro-ir-1.1.1.tar.gz
(139.4 kB
view hashes)
Built Distribution
Close
Hashes for experimaestro_ir-1.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b46c21209c5f72370a633448a2e8336bf7f02ba106c113ddd82107bb396c0096 |
|
MD5 | e49df36752943496435afb78e8fcb6b6 |
|
BLAKE2b-256 | 3828432641afeb7b708f18232f5b413f126f8ff3959e45e71ae4569e09822d50 |