A library for managing LLM models
Project description
Description
ModelhubClient: A Python client for the Modelhub API
Installation
pip install puyuan_modelhub --user
Usage
ModelhubClient
Initialize a client
from modelhub import ModelhubClient
client = ModelhubClient(
host="https://xxxx.com/api/",
user_name="xxxx",
user_password="xxxx",
model="xxx", # Optional
)
get supported models
client.supported_models
Get model supported params
client.get_supported_params("Minimax")
perform a chat query
response = client.chat(
query,
model="xxx", # Optional(use model in client construction)
history=history,
parameters=dict(
key1=value1,
key2=value2
)
)
Get model embeddings
client.get_embeddings(["你好", "Hello"], model="m3e")
Response structure
generated_text: response_text from model
history: generated history
details: generation details. Include tokens used, request duration, ...
History can be only used with ChatGLM3 now.
BaseMessage is the unit of history.
# import some pre-defined message types
from modelhub.common.types import SystemMessage, AIMessage, UserMessage
# construct history of your own
history = [
SystemMessage(content="xxx", other_value="xxxx"),
UserMessage(content="xxx", other="xxxx"),
]
VLMClient
Initailize a vlm client
from modelhub import VLMClient
client = VLMClient(...)
client.chat(prompt=..., image_path=..., parameters=...)
Chat with model
VLM Client chat add image_path
param to Modelhub Client and other params are same.
client.chat("Hello?", image_path="xxx", model="m3e")
Examples
Use ChatCLM3 for tools calling
from modelhub import ModelhubClient, VLMClient
from modelhub.common.types import SystemMessage
client = ModelhubClient(
host="https://xxxxx/api/",
user_name="xxxxx",
user_password="xxxxx",
)
tools = [
{
"name": "track",
"description": "追踪指定股票的实时价格",
"parameters": {
"type": "object",
"properties": {"symbol": {"description": "需要追踪的股票代码"}},
"required": ["symbol"],
},
},
{
"name": "text-to-speech",
"description": "将文本转换为语音",
"parameters": {
"type": "object",
"properties": {
"text": {"description": "需要转换成语音的文本"},
"voice": {"description": "要使用的语音类型(男声、女声等)"},
"speed": {"description": "语音的速度(快、中等、慢等)"},
},
"required": ["text"],
},
},
]
# construct system history
history = [
SystemMessage(
content="Answer the following questions as best as you can. You have access to the following tools:",
tools=tools,
)
]
query = "帮我查询股票10111的价格"
# call ChatGLM3
response = client.chat(query, model="ChatGLM3", history=history)
history = response["history"]
print(response["generated_text"])
Output:
{"name": "track", "parameters": {"symbol": "10111"}}
# generate a fake result for track function call
result = {"price": 1232}
res = client.chat(
json.dumps(result),
parameters=dict(role="observation"), # Tell ChatGLM3 this is a function call result
model="ChatGLM3",
history=history,
)
print(res["generated_text"])
Output:
根据API调用结果,我得知当前股票的价格为1232。请问您需要我为您做什么?
Contact
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
puyuan-modelhub-1.0.9.tar.gz
(6.7 kB
view hashes)
Built Distribution
Close
Hashes for puyuan_modelhub-1.0.9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89f0ffa39ec60a01982f411d0765fc0cb3fe5aec753cc3d061f9344bc3c3fd2c |
|
MD5 | f60916927b4cc9236d7d040765415f15 |
|
BLAKE2b-256 | 8a8bda5bda8032241e8dd5457ec14c4b011b6d5ad8878774ce985f5057b1318b |