Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
test_connections: true
connect_timeout: 30
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.23.tar.gz
(48.7 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.23.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b0879245e180a483a1ae12f6ece01fcb47b50f1040eac1a17a5b89d900640a49 |
|
MD5 | 11901bd9f7c2628c0ec880bb128b18ff |
|
BLAKE2b-256 | bd13a4bcc61b9961d16aaa1c81b5d6cd79f288d264277c3a34ca207823dc414a |
Close
Hashes for caikit_tgis_backend-0.1.23-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6a4e34c129d5b9e5752400d75d443374f2c5305c89b7a6afcf40c4df9d34795 |
|
MD5 | 39bf124f26078a83f1ceebf041b6ccec |
|
BLAKE2b-256 | f039de1d4737c891305878f8b01685b777bedc49075e3750a5b3fa359e0adc2f |