Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
connect_timeout: 30
test_connections: true
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit_tgis_backend-0.1.32.tar.gz
(51.4 kB
view hashes)
Built Distribution
Close
Hashes for caikit_tgis_backend-0.1.32.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6296dcb21badb747c1a8fb32c34accdb8b97e06e4b787acf7ffaf3f1b68c0c41 |
|
MD5 | 2ab191e05ef1b0e8f2769c6bc4fb73eb |
|
BLAKE2b-256 | 1e9e9716f58fb7f26ac08acb8c46a765879be27d88dad019a748038c34c199db |
Close
Hashes for caikit_tgis_backend-0.1.32-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87dd14e13cfdfb114bb60b304afe46499eddf8d86b74498e8e560c563d315b61 |
|
MD5 | 2088d9c52a3440d331d958025927ba2c |
|
BLAKE2b-256 | 8334a863d35506740b93cb423093b24b41324532f137ca700e7a31e06dfd7a1a |