Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
test_connections: true
connect_timeout: 30
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.29.tar.gz
(49.5 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.29.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d84c1352892332da8d01ae688436d2e0cb4f021df44217fa4b926edf66cb918 |
|
MD5 | a3a473259e923a3bd7055c36734fbd9f |
|
BLAKE2b-256 | aa9f088f81d3d85bb6c41ebb872785fa8e6ddfe69e7033e03b4c1a5625c4e4e6 |
Close
Hashes for caikit_tgis_backend-0.1.29-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b1bee0b7e5b1e54e72fa3e216d7b69933055527fc390c10cf18b5f3b208da14d |
|
MD5 | c94aca6e2ce47c4c5a7ab02e1577b37c |
|
BLAKE2b-256 | 3d5d5027715938f3af225549b980af28527942d983018824813e86dfc607d1cf |