Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
connect_timeout: 30
test_connections: true
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.31.tar.gz
(50.3 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.31.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | afdc8003ac83275abc958ab25fd5ae2b7b9c542fecfc1e5a9f49e932b707b63f |
|
MD5 | 02b644284987ba888386c494cbfb813e |
|
BLAKE2b-256 | 5ddaadd172ceb064ff504f815ed03140a95a1fa99a51e1c1b0116bcc3d4d3b60 |
Close
Hashes for caikit_tgis_backend-0.1.31-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4e7b7b442e74c7d202814b2a83333d95a34d3f32852d2fa6ff45b0c14c5718ab |
|
MD5 | e249f89da92162f2f8d2fca845881258 |
|
BLAKE2b-256 | a96ec6b6fb5cf119724a8ad82e2c85f3219f5d91782f26f48e27d93ae786e436 |