Inference locally.
Project description
LLM Inference, 开箱即用的本地 LLM 推理 SDK
安装
-
pip
pip install git+https://gitlab.changdu.ltd/060270/llm_inference.git@main -
uv
uv add git+https://gitlab.changdu.ltd/060270/llm_inference.git@main
主要功能
-
Seed-X 机器翻译
from vllm_inference.translation.seed_x import translate
translation = translate('你好, 我叫吴子豪!', 'fr', max_tokens=2048, temperature=.0, top_p=.01)
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
vllm_inference-0.0.16.tar.gz
(2.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file vllm_inference-0.0.16.tar.gz.
File metadata
- Download URL: vllm_inference-0.0.16.tar.gz
- Upload date:
- Size: 2.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ba9b0fce5464bcd1d99504c5cb00ae2816d0fc9983c294b09c4dfe9450bafc8d
|
|
| MD5 |
8b2866fe481171870d7cf180ef6f09a9
|
|
| BLAKE2b-256 |
eb382a992ad448a41539d91ef71a490be79d243bbdcc0b70ccb70b9065b48794
|
File details
Details for the file vllm_inference-0.0.16-py3-none-any.whl.
File metadata
- Download URL: vllm_inference-0.0.16-py3-none-any.whl
- Upload date:
- Size: 3.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3aa931abb83e6f4ac9534ba0a45515e4316b36f2f18712976b6b15c30d822d98
|
|
| MD5 |
e41fd792f78765840b75ae7dffb02c64
|
|
| BLAKE2b-256 |
ce8ccd20b68a24bb64bdfb123b92c0b1a2c75220d8208a07ea312a65823c6088
|
