llama-index postprocessor xinference rerank integration
Project description
LlamaIndex Postprocessor Integration: Xinference Rerank
Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models.
You can find a list of built-in rerank models in Xinference from its document Rerank Models
To learn more about Xinference in general, visit https://inference.readthedocs.io/en/stable/models/model_abilities/rerank.html
Installation
pip install llama-index-postprocessor-xinference-rerank
Usage
Parameters Description:
model
: Model uid not model name, sometimes they may be the same (e.g.,bge-reranker-base
).base_url
: base url of Xinference (e.g.,http://localhost:9997
).top_n
: Top n nodes to return from reranker. (default 5).
Nodes Rerank Example
from llama_index.postprocessor.xinference_rerank import XinferenceRerank
xi_model_uid = "xinference model uid"
xi_base_url = "xinference base url"
xi_rerank = XinferenceRerank(
top_n=5,
model=xi_model_uid,
base_url=xi_base_url,
)
def test_rerank_nodes(nodes, query_str):
response = xi_rerank.postprocess_nodes(nodes, query_str)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_postprocessor_xinference_rerank-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0998354785127c74ac78b5d87d5991d51834b53591cd2ff8b84cd3029e20b4f1 |
|
MD5 | 3c6016ca6d9ae6df97995f255b16f14b |
|
BLAKE2b-256 | 5f2460399a92a696129e1bd630ebd294c50e056804cd22721165fe26febfdc2a |
Close
Hashes for llama_index_postprocessor_xinference_rerank-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3cd97d9e46e15b3a5ea0303eaa14bdf2236f5796f60410d6facfaa4c1486d4bf |
|
MD5 | d35dc5f024f82db901fe41922f36ed3e |
|
BLAKE2b-256 | a6d6c0d0629cdc50fac17adab48a346c8470a84240ddb9b6f79e351ad443d9c8 |