Skip to main content

PyTerrier

Project description

Continuous Testing PyPI version Documentation Status

PyTerrier

A Python API for Terrier - v.0.13

Installation

The easiest way to get started with PyTerrier is to use one of our Colab notebooks - look for the Open In Colab badges below.

Linux or Google Colab or Windows

  1. pip install python-terrier
  2. You may need to set JAVA_HOME environment variable if Pyjnius cannot find your Java installation.

macOS

  1. You need to hava Java installed. Pyjnius/PyTerrier will pick up the location automatically.
  2. pip install python-terrier

Indexing

PyTerrier has a number of useful classes for creating indices:

  • You can create an index from TREC formatted collection using TRECCollectionIndexer.
  • For TXT, PDF, Microsoft Word files, etc files you can use FilesIndexer.
  • For any abitrary iterable dictionaries or a Pandas Dataframe, you can use IterDictIndexer.

See the indexing documentation, or the examples in the indexing notebook Open In Colab

Retrieval and Evaluation

topics = pt.io.read_topics(topicsFile)
qrels = pt.io.read_qrels(qrelsFile)
BM25_r = pt.terrier.Retriever(index, wmodel="BM25")
res = BM25_r.transform(topics)
pt.Evaluate(res, qrels, metrics = ['map'])

See also the retrieval documentation, or the worked example in the retrieval and evaluation notebook Open In Colab

Experiment - Perform Retrieval and Evaluation with a single function

PyTerrier provides an Experiment function, which allows to compare multiple retrieval approaches on the same queries & relevance assessments:

pt.Experiment([BM25_r, PL2_r], topics, qrels, ["map", "ndcg"])

There is a worked example in the experiment notebook Open In Colab

Pipelines

PyTerrier makes it easy to develop complex retrieval pipelines using Python operators such as >> to chain different retrieval components. Each retrieval approach is a transformer, having one key method, transform(), which takes a single Pandas dataframe as input, and returns another dataframe. Two examples might encapsulate applying the sequential dependence model, or a query expansion process:

sdm_bm25 = pt.rewrite.SDM() >> pt.terrier.Retriever(indexref, wmodel="BM25")
bo1_qe = BM25_r >> pt.rewrite.Bo1QueryExpansion() >> BM25_r

There is documentation on transformer operators as well as example pipelines show other common use cases. For more information, see the PyTerrier data model.

Neural Reranking and Dense Retrieval

PyTerrier has additional plugins for BERT (through OpenNIR), T5, ColBERT, doc2query and many more...

  • Pyterrier_DR: [Github] - single-representation dense retrieval
  • PyTerrier_ColBERT: [Github] - mulitple-representation dense retrieval and/or neural reranking
  • PyTerrier_PISA: [Github] - fast in-memory indexing and retrieval using PISA
  • PyTerrier_T5: [Github] - neural reranking: monoT5, duoT5
  • PyTerrier_GenRank [Github] - generative listwise reranking: RankVicuna, RankZephyr
  • PyTerrier_doc2query: [Github] - neural augmented indexing
  • PyTerrier_SPLADE: [Github] - neural augmented indexing

Older plugins include:

You can see examples of how to use these, including notebooks that run on Google Colab, in the contents of our Search Solutions 2022 tutorial.

Learning to Rank

Complex learning to rank pipelines, including for learning-to-rank, can be constructed using PyTerrier's operator language. For example, to combine two features and make them available for learning, we can use the ** operator.

two_features = BM25_r >> ( 
  pt.terrier.Retriever(indexref, wmodel="DirichletLM") ** 
  pt.terrier.Retriever(indexref, wmodel="PL2") 
)

See also the learning to rank documentation, as well as the worked examples in the learning-to-rank notebook Open In Colab. Some pipelines can be automatically optimised - more detail about pipeline optimisation are included in our ICTIR 2020 paper.

Dataset API

PyTerrier allows simple access to standard information retrieval test collections through its dataset API, which can download the topics, qrels, corpus or, for some test collections, a ready-made Terrier index.

topics = pt.get_dataset("trec-robust-2004").get_topics()
qrels = pt.get_dataset("trec-robust-2004").get_qrels()
pt.Experiment([BM25_r, PL2_r], topics, qrels, eval_metrics)

You can index datasets that include a corpus using IterDictIndexer and get_corpus_iter:

dataset = pt.get_dataset('irds:cord19/trec-covid')
indexer = pt.IterDictIndexer('./cord19-index')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=('title', 'abstract'))

You can use pt.list_datasets() to see available test collections - if your favourite test collection is missing, you can submit a Pull Request.

All datasets from the ir_datasets package are available under the irds: prefix. E.g., use pt.datasets.get_dataset("irds:medline/2004/trec-genomics-2004") to get the TREC Genomics 2004 dataset. A full catalogue of ir_datasets is available here.

Index API

All of the standard Terrier Index API can be access easily from Pyterrier.

For instance, accessing term statistics is a single call on an index:

index.getLexicon()["circuit"].getDocumentFrequency()

There are lots of examples in the index API notebook Open In Colab

Documentation

More documentation for PyTerrier is available at https://pyterrier.readthedocs.io/en/latest/.

Open Source Licence

PyTerrier is subject to the terms detailed in the Mozilla Public License Version 2.0. The Mozilla Public License can be found in the file LICENSE.txt. By using this software, you have agreed to the licence.

Citation Licence

The source and binary forms of PyTerrier are subject to the following citation license:

By downloading and using PyTerrier, you agree to cite at the undernoted paper describing PyTerrier in any kind of material you produce where PyTerrier was used to conduct search or experimentation, whether be it a research paper, dissertation, article, poster, presentation, or documentation. By using this software, you have agreed to the citation licence.

Declarative Experimentation in Information Retrieval using PyTerrier. Craig Macdonald and Nicola Tonellotto. In Proceedings of ICTIR 2020.

@inproceedings{pyterrier2020ictir,
    author = {Craig Macdonald and Nicola Tonellotto},
    title = {Declarative Experimentation inInformation Retrieval using PyTerrier},
    booktitle = {Proceedings of ICTIR 2020},
    year = {2020}
}

Credits

  • Alex Tsolov, University of Glasgow
  • Craig Macdonald, University of Glasgow
  • Nicola Tonellotto, University of Pisa
  • Arthur Câmara, TU Delft
  • Alberto Ueda, Federal University of Minas Gerais
  • Sean MacAvaney, Georgetown University/University of Glasgow
  • Chentao Xu, University of Glasgow
  • Sarawoot Kongyoung, University of Glasgow
  • Zhan Su, Copenhagen University
  • Marcus Schutte, TU Delft
  • Lukas Zeit-Altpeter, Friedrich Schiller University Jena

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python_terrier-0.13.1.tar.gz (193.6 kB view details)

Uploaded Source

Built Distribution

python_terrier-0.13.1-py3-none-any.whl (167.5 kB view details)

Uploaded Python 3

File details

Details for the file python_terrier-0.13.1.tar.gz.

File metadata

  • Download URL: python_terrier-0.13.1.tar.gz
  • Upload date:
  • Size: 193.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for python_terrier-0.13.1.tar.gz
Algorithm Hash digest
SHA256 828679a955ff1fc031789851365187fc5bd603cebfe2573c5679427d4e77a8d3
MD5 1c56fc589f62cb35f04d2f1d8bf0f203
BLAKE2b-256 08da8d8791e1f0e106cccd900c7bde1fec9dcec2a1abb5ce1c4b3df1f90809ae

See more details on using hashes here.

File details

Details for the file python_terrier-0.13.1-py3-none-any.whl.

File metadata

File hashes

Hashes for python_terrier-0.13.1-py3-none-any.whl
Algorithm Hash digest
SHA256 4aeae14c43dc87a9e1d44b6aa656a8d38bd56a48f341f8d65e5db0974052ed73
MD5 2908bb17ce27a83c9e3dbf3ab93f2489
BLAKE2b-256 f2a9a94212fc19178d63154d195723e046b924998d05b4e04856a23a617eabda

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page