A conversational passage retrieval toolkit
Project description
Chatty Goose
Multi-stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
Installation
-
Make sure Java 11+ and Python 3.6+ are installed
-
Install the
chatty-goose
PyPI module
pip install chatty-goose
- If you are using T5 or BERT, make sure to install PyTorch 1.5+ using your specific platform instructions
Quickstart Guide
The following example shows how to initialize a searcher and build a CQR agent from scratch using HQE and T5 as first-stage retrievers, and a BERT reranker. To see a working example agent, see chatty_goose/agents/cqragent.py.
First, load a searcher
from pyserini.search import SimpleSearcher
# Option 1: load a prebuilt index
searcher = SimpleSearcher.from_prebuilt_index("INDEX_NAME_HERE")
# Option 2: load a local Lucene index
searcher = SimpleSearcher("PATH_TO_INDEX")
searcher.set_bm25(0.82, 0.68)
Next, initialize one or more first-stage CQR retrievers
from chatty_goose.cqr import HQE, T5_NTR
from chatty_goose.settings import HQESettings, T5Settings
hqe = HQE(searcher, HQESettings())
t5 = T5_NTR(T5Settings())
Load a reranker
from chatty_goose.util import build_bert_reranker
reranker = build_bert_reranker()
Create a new RetrievalPipeline
from chatty_goose.pipeline import RetrievalPipeline
rp = RetrievalPipeline(searcher, [hqe, t5], searcher_num_hits=10, reranker=reranker)
And we're done! Simply call rp.retrieve(query)
to retrieve passages, or call rp.reset_history()
to reset the conversational history of the retrievers.
Running Experiments
-
Clone the repo and all submodules (
git submodule update --init --recursive
) -
Clone and build Anserini for evaluation tools
-
Install dependencies
pip install -r requirements.txt
- Follow the instructions under docs/cqr_experiments.md to run experiments using HQE, T5, or fusion.
Example Agent
To run an interactive conversational search agent with ParlAI, simply run cqragent.py
. By default, we use the CAsT 2019 pre-built Pyserini index, but it is possible to specify other indexes using the --from_prebuilt
flag. See the file for other possible arguments:
python -m chatty_goose.agents.cqragent
Alternatively, run the agent using ParlAI's command line interface:
python -m parlai interactive --model chatty_goose.agents.cqragent:ChattyGooseAgent
We also provide instructions to deploy the agent to Facebook Messenger using ParlAI under examples/messenger
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for chatty_goose-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72dca5e0f27e36ccbbf117565bfc454a875b4a26d551a0d8abdea7bdd51c8697 |
|
MD5 | 0d01586e04e51d474f5786b7464f68e1 |
|
BLAKE2b-256 | 4da8dbaac0212e95a29571131421ec15a8fcbc2f6db46538d2681c35734523d8 |