Caikit module backend for models run in TGIS
Project description
Caikit Text Generation Inference Service (TGIS) Backend
This project provides a Caikit module backend that manages models run in TGIS
Configuration
Sample configuration using the MULTI
finder and a remote TGIS
backend):
runtime:
library: caikit_nlp
local_models_dir: /path/to/models
lazy_load_local_models: true
model_management:
finders:
default:
type: MULTI
config:
finder_priority:
- local
- tgis-auto
initializers:
default:
type: LOCAL
config:
backend_priority:
- type: TGIS
config:
connection:
hostname: "localhost:8033"
test_connections: true
connect_timeout: 30
log:
formatter: pretty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
caikit-tgis-backend-0.1.28.tar.gz
(48.8 kB
view hashes)
Built Distribution
Close
Hashes for caikit-tgis-backend-0.1.28.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5502943b4ee2409e95cebc313b8a869691cc30534b39195001f449290b716cdf |
|
MD5 | cdd3590671624d2941185c1d1e2f5604 |
|
BLAKE2b-256 | cf1d531dc2641e665f9109bd2da993f1f9ed142f30ce691fb0c7ecac43b219fd |
Close
Hashes for caikit_tgis_backend-0.1.28-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 150e9d80483f619fbc45e669f56a1336fafa042ed9624a2937bc9c19cda234b1 |
|
MD5 | f0a139fbfc4eac2759b1e85f4af434b3 |
|
BLAKE2b-256 | 111da0b0e340e015d4b2422e98bb51afbc0bff3d181c7da14e88904361b279ba |