Inference locally.
Project description
LLM Inference, 开箱即用的本地 LLM 推理 SDK
安装
-
pip
pip install git+https://gitlab.changdu.ltd/060270/llm_inference.git@main -
uv
uv add git+https://gitlab.changdu.ltd/060270/llm_inference.git@main
主要功能
-
Seed-X 机器翻译
from vllm_inference.translation.seed_x import translate
translation = translate('你好, 我叫吴子豪!', 'fr', max_tokens=2048, temperature=.0, top_p=.01)
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
vllm_inference-0.0.19.tar.gz
(3.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file vllm_inference-0.0.19.tar.gz.
File metadata
- Download URL: vllm_inference-0.0.19.tar.gz
- Upload date:
- Size: 3.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8541a678ec672f9b4ca1fafd272519b10c5f879a108916276e63ca01f0a4b577
|
|
| MD5 |
7919e0a77c1de113a76c7888a6c89eaa
|
|
| BLAKE2b-256 |
f11048ff9cd17cfac56a855a1c049af7863eabd09b95732288aa0a0613516cb0
|
File details
Details for the file vllm_inference-0.0.19-py3-none-any.whl.
File metadata
- Download URL: vllm_inference-0.0.19-py3-none-any.whl
- Upload date:
- Size: 4.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8153c764c9a5d9df094d9f75f39b94224ed51585cd7abe26c81715dea7473505
|
|
| MD5 |
53b6af836149cb1fe17ca2b9eea542cc
|
|
| BLAKE2b-256 |
c69ad64b778947bc79d48b4c966ec998eb200738f650331f8fe468fc0e6c311b
|
