FlagEmbedding model worker for fastchat.
Project description
fs-FlagEmbedding-worker
FlagEmbedding model worker for fastchat.
Install
pip install fschat-FlagEmbedding-worker
Usage
启动embedding模型工作者
python -m fschat_flagembedding_worker.serve.flagmodel_worker --controller-address http://localhost:21001 --model-path=/models/BAAI/bge-large-zh-v1.5 --port=22002 --worker-address=http://localhost:22002 --debug
启动reranker模型工作者
python -m fschat_flagembedding_worker.serve.flagreranker_worker --controller-address http://localhost:21001 --model-path=/models/BAAI/bge-reranker-large --port=22003 --worker-address=http://localhost:22003 --debug
使用embedding模型
import os
from langchain_openai.embeddings import OpenAIEmbeddings
from openai.resources.embeddings import Embeddings
API_SECRET_KEY = "QufTZCA5Y1zIaC3GxNBUn2Q0vW1NITKm"
BASE_URL = "http://localhost:8000/v1"
EMBEDDING_MODEL_NAME = "bge-large-zh-v1.5"
RERANKER_MODEL_NAME = "bge-reranker-large"
os.environ["OPENAI_API_KEY"] = API_SECRET_KEY
os.environ["OPENAI_API_BASE"] = BASE_URL
embd = OpenAIEmbeddings(
model=EMBEDDING_MODEL_NAME,
check_embedding_ctx_length=False,
tiktoken_enabled=False,
)
r1 = embd.embed_query("hello world")
print(r1)
输出:
[-0.02532958984375, -0.0016546249389648438, ..., 0.018463134765625, 0.0161895751953125]
使用reranker模型
- openapi并没有提供reranker响应的接口,这里我们将使用embedding接口来获取reranker的响应。
- 但对输入参数进行了约定,要求格式如下:[query, passage1, query, passage2, ...]。
- 特别需要注意的是,在启动openai_api服务时,需要添加额外的环境变量:FASTCHAT_WORKER_API_EMBEDDING_BATCH_SIZE。
openai api server启动案例:
#!/bin/bash
export FASTCHAT_WORKER_API_EMBEDDING_BATCH_SIZE=128
/opt/conda/bin/python -m fastchat.serve.openai_api_server --host 0.0.0.0 -api-keys MRpta0LlV0SrxpYBgbecyyF47Pvt263O
reranker模型使用案例:
import os
from langchain_openai.embeddings import OpenAIEmbeddings
from openai.resources.embeddings import Embeddings
API_SECRET_KEY = "QufTZCA5Y1zIaC3GxNBUn2Q0vW1NITKm"
BASE_URL = "http://localhost:8000/v1"
EMBEDDING_MODEL_NAME = "bge-large-zh-v1.5"
RERANKER_MODEL_NAME = "bge-reranker-large"
os.environ["OPENAI_API_KEY"] = API_SECRET_KEY
os.environ["OPENAI_API_BASE"] = BASE_URL
reranker = OpenAIEmbeddings(
model=RERANKER_MODEL_NAME,
check_embedding_ctx_length=False,
tiktoken_enabled=False,
)
query = "hello"
passages = ["hi", "world", "yes", "how are you?"]
messages = []
for passage in passages:
messages += [query, passage]
r1 = reranker.embed_query(messages)
print(r1)
输出:
[5.96577262878418, -3.2970359325408936, 0.9453534483909607, 0.5078737735748291]
Releases
v0.1.1
- First release.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for fschat-FlagEmbedding-worker-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e3d0563c7ac1ea730e00d96b5a2edfc830bb965126134da6e0eb9377909cd23 |
|
MD5 | 93eca5dc330e322ebb1a7d8834b2cbce |
|
BLAKE2b-256 | 43d80e5bd19867789e81e786c3ef80dbf499326a03ced65cf063cf2bde0a0571 |
Close
Hashes for fschat_FlagEmbedding_worker-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 718e92109734b303a74e01da346d7a301b932fc0e9b54ac23d94ff3f673d1ec9 |
|
MD5 | 8c3bdccc82da72f3b4d74076bd2d09e2 |
|
BLAKE2b-256 | 125a63d387de6f50701c4b1fdb6e4c896ca4065da7567431367765c4ccf281ad |