A library for managing LLM models
Project description
Description
ModelhubClient: A Python client for the Modelhub API
Installation
pip install puyuan_modelhub --user
Usage
ModelhubClient
from modelhub import ModelhubClient
client = ModelhubClient(
host="https://xxxx.com/api/",
user_name="xxxx",
user_password="xxxx",
model="xxx", # Optional
)
response = client.chat(
query,
model="xxx", # Optional(use model in client construction)
history=history,
parameters=dict(
key1=value1,
key2=value2
)
)
Response format:
generated_text: response_text from model
history: generated history
details: generation details. Include tokens used, request duration, ...
History can be only used with ChatGLM3 now.
BaseMessage is the unit of history.
# import some pre-defined message types
from modelhub.common.types import SystemMessage, AIMessage, UserMessage
# construct history of your own
history = [
SystemMessage(content="xxx", other_value="xxxx"),
UserMessage(content="xxx", other="xxxx"),
]
VLMClient
from modelhub import VLMClient
client = VLMClient(...)
client.chat(prompt=..., image_path=..., parameters=...)
Initialize a ModelhubClient
from modelhub import ModelhubClient
client = ModelhubClient(
host="http://*****:*****/", user_name="****", user_password="*****"
)
Get supported Models
client.supported_models
Get model supported params
client.get_supported_params("Minimax")
Chat with model
client.chat("Hello?", model="m3e")
Get model embeddings
client.get_embeddings(["你好", "Hello"], model="m3e")
Examples
Use ChatCLM3 for tools calling
from modelhub import ModelhubClient, VLMClient
from modelhub.common.types import SystemMessage
client = ModelhubClient(
host="https://modelhub.puyuan.tech/api/",
user_name="admin",
user_password="puyuan.abc",
)
tools = [
{
"name": "track",
"description": "追踪指定股票的实时价格",
"parameters": {
"type": "object",
"properties": {"symbol": {"description": "需要追踪的股票代码"}},
"required": ["symbol"],
},
},
{
"name": "text-to-speech",
"description": "将文本转换为语音",
"parameters": {
"type": "object",
"properties": {
"text": {"description": "需要转换成语音的文本"},
"voice": {"description": "要使用的语音类型(男声、女声等)"},
"speed": {"description": "语音的速度(快、中等、慢等)"},
},
"required": ["text"],
},
},
]
# construct system history
history = [
SystemMessage(
content="Answer the following questions as best as you can. You have access to the following tools:",
tools=tools,
)
]
query = "帮我查询股票10111的价格"
# call ChatGLM3
response = client.chat(query, model="ChatGLM3", history=history)
history = response["history"]
print(response["generated_text"])
Output:
{"name": "track", "parameters": {"symbol": "10111"}}
# generate a fake result for track function call
result = {"price": 1232}
res = client.chat(
json.dumps(result),
parameters=dict(role="observation"), # Tell ChatGLM3 this is a function call result
model="ChatGLM3",
history=history,
)
print(res["generated_text"])
Output:
根据API调用结果,我得知当前股票的价格为1232。请问您需要我为您做什么?
Contact
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
puyuan-modelhub-1.0.4.tar.gz
(6.4 kB
view hashes)
Built Distribution
Close
Hashes for puyuan_modelhub-1.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 74467d442cec2be7a5d83138c0e50cf82037222d358202b485c36bd161b22bf4 |
|
MD5 | 6c6b3ae7a1964afa772129be962f74d1 |
|
BLAKE2b-256 | e82bf037c718e8fcdccaecae40cc04e2cacd9fa793313a75cbcaac93a0d38dac |