Imitater
A unified language model server built upon vllm and infinity.
Usage
Install
pip install -e .
Launch Server
imitater -c config/example.yaml
Show configuration instruction.
Add an OpenAI model
- name: OpenAI model name
- token: OpenAI token
Add a chat model
- name: Display name
- path: Model name on hub or local model path
- device: Device IDs
- port: Port ID
- maxlen: Maximum model length (optional)
- agent_type: Agent type (optional) {react, aligned}
- template: Template jinja file (optional)
- gen_config: Generation config folder (optional)
Add an embedding model
- name: Display name
- path: Model name on hub or local model path
- device: Device IDs (does not support multi-gpus)
- port: Port ID
- batch_size: Batch size (optional)
Test Server
python tests/test_openai.py -c config/example.yaml
Roadmap
- Response choices.
- Rerank model support.
1 maintainer
Metadata
1 maintainer
Release files for imitater 0.2.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| imitater-0.2.4.tar.gz | 20.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| imitater-0.2.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 45.7 kB
Release files / imitater-0.2.4.tar.gz
| Download URL | imitater-0.2.4.tar.gz |
|---|---|
| Size | 20.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1639829832dabe14196d6f653de4208c67767c7d2e29a4d21d05e8cec160f389
|
|
BLAKE2b-256 checksum How to use checksums |
08bbe381e84631c56f2ad21609ec50995daf9796590eae6896c94e8521375843
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.13
|
Release files / imitater-0.2.4-py3-none-any.whl
| Download URL | imitater-0.2.4-py3-none-any.whl |
|---|---|
| Size | 25.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
34012bf16e724955bda91f78f7ec68303d7b6a236f3f0d9e81ed5cf7c9b64f91
|
|
BLAKE2b-256 checksum How to use checksums |
fb9b492bb87c063dd65d27ebd2df0196023702a011c851d1db191131422b3c4d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.10.13
|
