llama-index postprocessor rankllm-rerank integration
Project description
LlamaIndex Postprocessor Integration: Rankllm-Rerank
RankLLM offers a suite of listwise rerankers, albeit with focus on open source LLMs finetuned for the task. Currently, RankLLM supports 2 of these models: RankZephyr (model="zephyr"
) and RankVicuna (model="vicuna"
). RankLLM also support RankGPT usage (model="gpt"
, gpt_model="VALID_OPENAI_MODEL_NAME"
).
Please pip install llama-index-postprocessor-rankllm-rerank
to install RankLLM rerank package.
Parameters:
- top_n: Top N nodes to return from reranking.
- model: Reranker model name/class (
zephyr
,vicuna
, orgpt
). - with_retrieval[Optional]: Perform retrieval before reranking with
Pyserini
. - step_size[Optional]: Step size of sliding window for reranking large corpuses.
- gpt_model[Optional]: OpenAI model to use (e.g.,
gpt-3.5-turbo
) ifmodel="gpt"
💻 Example Usage
pip install llama-index-core
pip install llama-index-llms-openai
pip install llama-index-postprocessor-rankllm-rerank
First, build a vector store index with llama-index.
index = VectorStoreIndex.from_documents(
documents,
)
To set up the retriever and reranker:
query_bundle = QueryBundle(query_str)
# configure retriever
retriever = VectorIndexRetriever(
index=index,
similarity_top_k=vector_top_k,
)
# configure reranker
reranker = RankLLMRerank(
top_n=reranker_top_n,
model=model,
with_retrieval=with_retrieval,
step_size=step_size,
gpt_model=gpt_model,
)
To run retrieval+reranking:
# retrieve nodes
retrieved_nodes = retriever.retrieve(query_bundle)
# rerank nodes
reranked_nodes = reranker.postprocess_nodes(
retrieved_nodes, query_bundle
)
🔧 Dependencies
Currently, RankLLM rerankers require CUDA
and for rank-llm
to be installed (pip install rank-llm
). The built-in retriever, which uses Pyserini, requires JDK11
, PyTorch
, and Faiss
.
castorini/rank_llm
Repository for prompt-decoding using LLMs (GPT3.5
, GPT4
, Vicuna
, and Zephyr
)
Website: http://rankllm.ai
Stars: 193
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2dca372ab36190be8b7338b0e83249d8e4881d3f23444cd9298f689c94175dcd |
|
MD5 | e88eb2a998f1f30ca5ed6a141de97648 |
|
BLAKE2b-256 | 114ce43c24843c3647c9d079884531798734496739b1b0fceb6146578e81be36 |
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c706f17c4d1b9030e538f64f082e6cafc1b9a9d38ff1c6d658d2953cae6d7a52 |
|
MD5 | 171c611adbd348540564bfdbfd3047fc |
|
BLAKE2b-256 | d00821be77232cda0c889f412dc398f1736f5b921d732a166fc7254f7aa6339c |