llama-index postprocessor rankllm-rerank integration
Project description
LlamaIndex Postprocessor Integration: Rankllm-Rerank
RankLLM offers a suite of listwise rerankers, albeit with focus on open source LLMs finetuned for the task. Currently, RankLLM supports 2 of these models: RankZephyr (model="zephyr"
) and RankVicuna (model="vicuna"
). RankLLM also support RankGPT usage (model="gpt"
, gpt_model="VALID_OPENAI_MODEL_NAME"
).
Please pip install llama-index-postprocessor-rankllm-rerank
to install RankLLM rerank package.
Parameters:
- top_n: Top N nodes to return from reranking.
- model: Reranker model name/class (
zephyr
,vicuna
, orgpt
). - with_retrieval[Optional]: Perform retrieval before reranking with
Pyserini
. - step_size[Optional]: Step size of sliding window for reranking large corpuses.
- gpt_model[Optional]: OpenAI model to use (e.g.,
gpt-3.5-turbo
) ifmodel="gpt"
💻 Example Usage
pip install llama-index-core
pip install llama-index-llms-openai
pip install llama-index-postprocessor-rankllm-rerank
First, build a vector store index with llama-index.
index = VectorStoreIndex.from_documents(
documents,
)
To set up the retriever and reranker:
query_bundle = QueryBundle(query_str)
# configure retriever
retriever = VectorIndexRetriever(
index=index,
similarity_top_k=vector_top_k,
)
# configure reranker
reranker = RankLLMRerank(
top_n=reranker_top_n,
model=model,
with_retrieval=with_retrieval,
step_size=step_size,
gpt_model=gpt_model,
)
To run retrieval+reranking:
# retrieve nodes
retrieved_nodes = retriever.retrieve(query_bundle)
# rerank nodes
reranked_nodes = reranker.postprocess_nodes(
retrieved_nodes, query_bundle
)
🔧 Dependencies
Currently, RankLLM rerankers require CUDA
and for rank-llm
to be installed (pip install rank-llm
). The built-in retriever, which uses Pyserini, requires JDK11
, PyTorch
, and Faiss
.
castorini/rank_llm
Repository for prompt-decoding using LLMs (GPT3.5
, GPT4
, Vicuna
, and Zephyr
)
Website: http://rankllm.ai
Stars: 193
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5516ce2c9e32e8ad1eaf73e0f5483e5039ee4c540d9fa0f5af461c0fc3e1ac0 |
|
MD5 | aa5961fa6ced7e4a06a380e5c12d11e0 |
|
BLAKE2b-256 | e13bc5a7a9c3efe3097c9d1d6d4aac867bbd06414a9503470fba28ca17fff5b6 |
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1e229f35a99c60619923e9bc276bbb161014d67486e301e4ea837c227082b53 |
|
MD5 | a68f206f4b3917e73b904d3074581c8f |
|
BLAKE2b-256 | 6a2f7f35fc8cd491797c3f759dd2319681ac497e443cea291e3f6965e436fd8b |