Efficient interpolation-based ranking on CPUs
Project description
Fast-Forward Indexes
This is the reference implementation of Fast-Forward indexes.
⚠ Important: As this library is still in its early stages, the API is subject to change! ⚠
Installation
Install the package via pip
:
pip install fast-forward-indexes
Getting Started
Using a Fast-Forward index is as simple as providing a TREC run with sparse scores:
from pathlib import Path
from fast_forward.encoder import TCTColBERTQueryEncoder
from fast_forward.index import InMemoryIndex, Mode
from fast_forward.ranking import Ranking
# choose a pre-trained query encoder
encoder = TCTColBERTQueryEncoder("castorini/tct_colbert-msmarco")
# load an index from disk into memory
index = InMemoryIndex.from_disk(Path("/path/to/index"), encoder, Mode.MAXP)
# load a sparse run (TREC format)
sparse_ranking = Ranking.from_file(Path("/path/to/sparse/run.tsv"))
# load all required queries
queries = {
"q1": "query 1",
"q2": "query 2",
# ...
"qn": "query n"
}
# compute the corresponding dense scores and interpolate
alpha = 0.2
result = index.get_scores(
sparse_ranking,
queries,
alpha=alpha,
cutoff=10,
early_stopping=True
)
# create a new TREC runfile with the interpolated ranking
result[alpha].save(Path("/path/to/interpolated/run.tsv"))
Documentation
A more detailed documentation is available here.
Examples
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
fast-forward-indexes-0.1.0.tar.gz
(13.7 kB
view hashes)
Built Distribution
Close
Hashes for fast-forward-indexes-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28ac7de866221d2f12591169472ab6a969f366fae31465ba64c64841dbb8d241 |
|
MD5 | 69a860fd24bfbbd382f23645b5eb0820 |
|
BLAKE2b-256 | 596c112ab5f6aa2ff87eab8b8c46f0807cb8a208aeaadd1c099da107828c8a1e |
Close
Hashes for fast_forward_indexes-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05240158731f67f655393e38f1ac1341662e4242266a6af0ea5a709ec03eef9d |
|
MD5 | 82a65f9b0cb0c6aa5ce9a32847c96025 |
|
BLAKE2b-256 | 878ee4a59ca70fc705d9302d9b0d967ef7885d3cc804122912cc2cf64473af2f |