llama-index embeddings xinference integration
Project description
LlamaIndex Embeddings Integration: Xinference
Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models.
You can find a list of built-in embedding models in Xinference from its document Embedding Models
To learn more about Xinference in general, visit https://inference.readthedocs.io/en/latest/
Installation
pip install llama-index-embeddings-xinference
Usage
Parameters Description:
model_uid
: Model uid not the model name, sometimes they may be the same (e.g.,bce-embedding-base_v1
).base_url
: base url of Xinference (e.g.,http://localhost:9997
).timeout
: request timeout set (default 60s).prompt
: Text to embed.
Text Embedding Example
from llama_index.embeddings.xinference import XinferenceEmbedding
xi_model_uid = "xinference model uid"
xi_base_url = "xinference base url"
xi_embed = XinferenceEmbedding(
model_uid=xi_model_uid,
base_url=xi_base_url,
timeout=60,
)
def text_embedding(prompt: str):
embeddings = xi_embed.get_query_embedding(prompt)
print(embeddings)
async def async_text_embedding(prompt: str):
embeddings = await xi_embed.aget_query_embedding(prompt)
print(embeddings)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_embeddings_xinference-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f24510d4f423ad724998af22d3b0e0a94308757be8b51ab6dfd43e2b3264307 |
|
MD5 | e4074aebc71e2dfb0df72d9f81ee8af8 |
|
BLAKE2b-256 | 59c932b3958f622f4887104e9b1d51c45f9a4181e4f894f05690fcd3b3e8d07b |
Close
Hashes for llama_index_embeddings_xinference-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3095536d3c7ce559d8c153b070f829a702eab6aaa53e859efae7234804ff2451 |
|
MD5 | cfba127c4ace2079f86e33fd918e8e5f |
|
BLAKE2b-256 | 5eb511d211c96eea1ff51c8cdf5fdf9760ac966f0a669cc1cab23fac9eb1ae31 |