Efficient interpolation-based ranking on CPUs
Project description
Fast-Forward Indexes
This is the implementation of Fast-Forward indexes.
Important: As this library is still in its early stages, the API is subject to change!
Installation
Install the package via pip
:
pip install fast-forward-indexes
Getting Started
Using a Fast-Forward index is as simple as providing a TREC run with retrieval scores:
from pathlib import Path
from fast_forward import OnDiskIndex, Mode, Ranking
from fast_forward.encoder import TCTColBERTQueryEncoder
# choose a pre-trained query encoder
encoder = TCTColBERTQueryEncoder("castorini/tct_colbert-msmarco")
# load an index on disk
ff_index = OnDiskIndex.load(Path("/path/to/index.h5"), encoder, Mode.MAXP)
# load a run (TREC format) and attach all required queries
first_stage_ranking = (
Ranking.from_file(Path("/path/to/input/run.tsv"))
.attach_queries(
{
"q1": "query 1",
"q2": "query 2",
# ...
"qn": "query n",
}
)
.cut(5000)
)
# compute the corresponding semantic scores
out = ff_index(first_stage_ranking)
# interpolate scores and create a new TREC runfile
first_stage_ranking.interpolate(out, 0.1).save(Path("/path/to/output/run.tsv"))
Documentation
A more detailed documentation is available here.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
fast-forward-indexes-0.2.0.tar.gz
(24.9 kB
view hashes)
Built Distribution
Close
Hashes for fast-forward-indexes-0.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82b53d2a05e8b04d920d2aab15f5544dd92c3326b26350a043d0236d111066a1 |
|
MD5 | ebab0e717f413f0bdc589308d95d4a51 |
|
BLAKE2b-256 | 0339273a00666b72af220d3009976e7d035e231e94426b2805375fd839cffe88 |
Close
Hashes for fast_forward_indexes-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4602a32dd6ca1c86a1f21c56f18aba13c25bb541ff4cae19d3d9da37638b7225 |
|
MD5 | ad768d9f78cbe697938144670ee55944 |
|
BLAKE2b-256 | 628ded155da84f0467cb6bf90e72ab24258290c843a48e7ea6ad48cb5223c16e |