Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
test_connections: true
connect_timeout: 30
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.25.tar.gz
(48.6 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.25.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4b8d5c38b8a8b3f0b4ec9af9d43366ab2beb0e52b654132fe475f3401e5858d |
|
MD5 | 0d28d427b59593ad7558fbf38360e5d9 |
|
BLAKE2b-256 | 175466fcb2e14cb8779c7aef8fd315760f8066ef73fb3806627bd8e1f7a61767 |
Close
Hashes for caikit_tgis_backend-0.1.25-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 869438f0a4d42c74480fccf461ecbeb55450445f6fa12b20ddec55a2aa426d99 |
|
MD5 | f3231dec2c08f83837ea937b2034f8a3 |
|
BLAKE2b-256 | 73ab899605eebc359c27a70e47bdc43f96fe5a1a9007380d6ab27d478e0a8944 |