No project description provided
Project description
Imitater
A unified language model server built upon vllm and infinity.
Usage
Install
pip install -U imitater
Launch Server
python -m imitater.service.app -c config/example.yaml
[!NOTE] Chat template is required for the chat models.
Use
export USE_MODELSCOPE_HUB=1
to download model from modelscope.
Test Server
python tests/test_openai.py -c config/example.yaml
Roadmap
- Response choices.
- Rerank model support.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
imitater-0.2.1.tar.gz
(19.3 kB
view hashes)
Built Distribution
imitater-0.2.1-py3-none-any.whl
(23.5 kB
view hashes)