llama-index postprocessor rankllm-rerank integration
Project description
LlamaIndex Postprocessor Integration: Rankllm-Rerank
RankLLM offers a suite of listwise rerankers, albeit with focus on open source LLMs finetuned for the task. Currently, RankLLM supports 2 of these models: RankZephyr (model="zephyr"
) and RankVicuna (model="vicuna"
).
Please pip install llama-index-postprocessor-rankllm-rerank
to install RankLLM rerank package.
💻 Example Usage
pip install llama-index-core
pip install llama-index-llms-openai
pip install llama-index-postprocessor-rankllm-rerank
First, build a vector store index with llama-index.
index = VectorStoreIndex.from_documents(
documents,
)
To set up the retriever and reranker:
query_bundle = QueryBundle(query_str)
# configure retriever
retriever = VectorIndexRetriever(
index=index,
similarity_top_k=vector_top_k,
)
# configure reranker
reranker = RankLLMRerank(
top_n=reranker_top_n, with_retrieval=with_retrieval,
model=model
)
To run retrieval+reranking:
# retrieve nodes
retrieved_nodes = retriever.retrieve(query_bundle)
# rerank nodes
reranked_nodes = reranker.postprocess_nodes(
retrieved_nodes, query_bundle
)
🔧 Dependencies
Currently, RankLLM rerankers require CUDA
and for rank-llm
to be installed (pip install rank-llm
). The built-in retriever, which uses Pyserini, requires JDK11
, PyTorch
, and Faiss
.
castorini/rank_llm
Repository for prompt-decoding using LLMs (GPT3.5
, GPT4
, Vicuna
, and Zephyr
)
Website: http://rankllm.ai
Stars: 193
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87db453599cd508f624bce082d7d40f5c50bdddce8703b407df4ad090f8b9847 |
|
MD5 | 4199e5e449db67a673d4bc501314c5d9 |
|
BLAKE2b-256 | 462af81c7ce28489ff8eb544099d57688a15ccccdb7fb1406af541b31a4612a8 |
Hashes for llama_index_postprocessor_rankllm_rerank-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad257b3c5680181dfc62118e820300380ea53cef0baa6ecde2bdf6c43020c213 |
|
MD5 | 3dc2f66cca76964c628e404b4a728ea7 |
|
BLAKE2b-256 | 4ffe174fe880b9b77ddd078831e2879f4710d0c2fcffed3f357429725b6540df |