Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
connect_timeout: 30
test_connections: true
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.30.tar.gz
(49.4 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.30.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 449b3a1b395749cc929f5e6e784c48b9fe19a794428ddfbf076dd0f2f23ddbe7 |
|
MD5 | f5f54e90ba248aea16a65f8462c64c07 |
|
BLAKE2b-256 | ec9bc587ff3c8708c2dfdefb70e90539b85753ae967b4825445b2eaf7907c979 |
Close
Hashes for caikit_tgis_backend-0.1.30-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | edff22d6b57745f01f457cf517e1d256a96de00d1f119f2030063a0eb81bc342 |
|
MD5 | 03e3d793368ba731e0ba94724334beb2 |
|
BLAKE2b-256 | 0e60598031c2a9b46e7afa9c54046c143fe77aaf80ad06b0f903601ab7fa2da1 |