puyuan-modelhub 1.0.3

A library for managing LLM models

Description

ModelhubClient: A Python client for the Modelhub API

Installation

Install from PyPI

pip install puyuan_modelhub

Install from source

git clone https://github.com/puyuantech/modelhub
python setup.py build bdist_wheel
pip install dist/*.whl

Usage

Client side

Initialize a ModelhubClient

from modelhub import ModelhubClient

client = ModelhubClient(
    host="http://*****:*****/", user_name="****", user_password="*****"
)

Get supported Models

client.supported_models

Get model supported params

client.get_supported_params("Minimax")

Chat with model

client.chat("Hello?", model="m3e")

Get model embeddings

client.get_embeddings(["你好", "Hello"], model="m3e")

Server side

Start server in 4 lines:

from modelhub.server import start_server
import yaml

# load config
config = yaml.xxxx
# start server
start_server(config)

config file example:

from modelhub.server import start_server
import yaml

# load config
config = yaml.xxxx
# start server
start_server(config)

Chat Params:

class ChatParams(BaseModel):
    prompt: str
    model: str
    auth: AuthParams
    stream: bool = True
    parameters: Dict[str, Any] = {}

Examples

Contact