Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
connect_timeout: 30
test_connections: true
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit_tgis_backend-0.1.33.tar.gz
(51.7 kB
view hashes)
Built Distribution
Close
Hashes for caikit_tgis_backend-0.1.33.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f51d44ed3b7b4ef3271332f6ea29a30626f778f70c56d298e39d076e06355ad6 |
|
MD5 | 53e7b3ed57f7188ce0c3dd7def512670 |
|
BLAKE2b-256 | 26f12e79e1651f0e3b02a6717c22616d702dd5c0a66965e82b906ff4e88a7120 |
Close
Hashes for caikit_tgis_backend-0.1.33-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df372920fd2ee2fd364256bfe1207070e9ee7c804f26b9ab5526a836a642a853 |
|
MD5 | 8c7cf5f0beec0eb80449a4a2e41a528c |
|
BLAKE2b-256 | 0cd4facb09d8e38d03e1f15df4e5c0491bdcc50bfb353b4ec2c35a19d376bd9e |