Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
connect_timeout: 30
test_connections: true
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit_tgis_backend-0.1.34.tar.gz
(53.3 kB
view hashes)
Built Distribution
Close
Hashes for caikit_tgis_backend-0.1.34.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b025fd30597c7667c0cc8c89beb8a22a7bbad759b450b2086fdeefd9392617c |
|
MD5 | 53570199dae4754b48c82c9ceaa78afe |
|
BLAKE2b-256 | 911b6b03cec1380bc902926f88d205c296e56bf1dcf94889122f7b571f86277b |
Close
Hashes for caikit_tgis_backend-0.1.34-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8a336597aed6cf4bbc5a47841d0b901c6106b092e4a116cd237c5727ed7380d |
|
MD5 | 97828e801d00cc8c46e7cb8b55b5b921 |
|
BLAKE2b-256 | e0ff40daaad3f5ac4a55326ccde3844a7c5f8cec58375bb71edfcd1c3db06ff3 |