Skip to main content

Efficient interpolation-based ranking on CPUs

Project description

Fast-Forward Indexes

This is the implementation of Fast-Forward indexes.

[!IMPORTANT] As this library is still in its early stages, the API is subject to change!

Installation

Install the package via pip:

pip install fast-forward-indexes

Getting Started

Using a Fast-Forward index is as simple as providing a TREC run with retrieval scores:

from pathlib import Path

from fast_forward import Ranking
from fast_forward.index import OnDiskIndex, Mode
from fast_forward.encoder import TCTColBERTQueryEncoder

# choose a pre-trained query encoder
encoder = TCTColBERTQueryEncoder("castorini/tct_colbert-msmarco")

# load an index on disk
ff_index = OnDiskIndex.load(Path("/path/to/index.h5"), encoder, mode=Mode.MAXP)

# load a run (TREC format) and attach all required queries
first_stage_ranking = (
    Ranking.from_file(Path("/path/to/input/run.tsv"))
    .attach_queries(
        {
            "q1": "query 1",
            "q2": "query 2",
            # ...
            "qn": "query n",
        }
    )
    .cut(5000)
)

# compute the corresponding semantic scores
out = ff_index(first_stage_ranking)

# interpolate scores and create a new TREC runfile
first_stage_ranking.interpolate(out, 0.1).save(Path("/path/to/output/run.tsv"))

Documentation

A more detailed documentation is available here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fast_forward_indexes-0.7.1.tar.gz (36.3 kB view details)

Uploaded Source

Built Distribution

fast_forward_indexes-0.7.1-py3-none-any.whl (37.1 kB view details)

Uploaded Python 3

File details

Details for the file fast_forward_indexes-0.7.1.tar.gz.

File metadata

  • Download URL: fast_forward_indexes-0.7.1.tar.gz
  • Upload date:
  • Size: 36.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for fast_forward_indexes-0.7.1.tar.gz
Algorithm Hash digest
SHA256 d3b79dc99a04c49875322b6388d6e931ead3c847703d7429f24764283f78fb62
MD5 6b3fa140b111f870fdec3687039b9fbe
BLAKE2b-256 0783c9f2211b690e8fb59ced13e22c1dd7ba36b82f32dfccd8de8755c3c7b4d5

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_forward_indexes-0.7.1.tar.gz:

Publisher: publish_pypi.yml on mrjleo/fast-forward-indexes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file fast_forward_indexes-0.7.1-py3-none-any.whl.

File metadata

File hashes

Hashes for fast_forward_indexes-0.7.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a4aa71e969c304f726416246eaa56f7a6b6306e573f6b2524f142d2e4bdeee1d
MD5 ec587592f63c65aafa066f98c6f2078c
BLAKE2b-256 227502e92622dd3cfae2f133d621efa21f76fcd80314540afb44011010ba3935

See more details on using hashes here.

Provenance

The following attestation bundles were made for fast_forward_indexes-0.7.1-py3-none-any.whl:

Publisher: publish_pypi.yml on mrjleo/fast-forward-indexes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page