Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
test_connections: true
connect_timeout: 30
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.27.tar.gz
(48.7 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.27.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa3cd3ee136d901a887ccfef8a9c0cf01d36bf42ea76cbb2bf414cc1df94f657 |
|
MD5 | cebc267b5ed16af6d89bc7ececd0c64f |
|
BLAKE2b-256 | a9494ebdce18dcc745a7dff3bb38ee9911f26d15d00ba907d4c31dbbeffffeeb |
Close
Hashes for caikit_tgis_backend-0.1.27-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 636e43450895f7532d7846cf97d3df4fdec90a741d00c5d552fc2d6d15190223 |
|
MD5 | a5f1df682763a627e3b23b89f878c6aa |
|
BLAKE2b-256 | d676ab6877c3608b432ee5cb56865a35a0fff903930c90a58c45fa9bb97a2512 |