Deploy LLMs to Kubernetes.
Project description
Flint
Deploy LLMs to Kubernetes.
v0.1.0 — early (alpha) release. Interfaces may still change; try it on a non-production cluster first. The GPU serving paths (vLLM, TGI) are best validated on your own hardware.
Flint is an open-source CLI that deploys large language models to your own Kubernetes cluster and routes traffic between versions. It uses your existing kubeconfig; there is no Flint server to operate.
Install
pip install flint-llm # the distribution is flint-llm; the CLI command is `flint`
Status
Alpha — v0.1.0. See CHANGELOG.md for what's included and docs/troubleshooting.md for common errors.
License
Apache 2.0. See LICENSE.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file flint_llm-0.1.0.tar.gz.
File metadata
- Download URL: flint_llm-0.1.0.tar.gz
- Upload date:
- Size: 41.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2e889e8403c01aa82465cb2ec6e2377d02099587cee084702d8ffab56792042e
|
|
| MD5 |
8ca6c27b9818798c67e444472ccb026a
|
|
| BLAKE2b-256 |
f4210ec45a55f124ef02056f81b69d057de68dc1b91c6af07ba0c7ca0fd951e3
|
File details
Details for the file flint_llm-0.1.0-py3-none-any.whl.
File metadata
- Download URL: flint_llm-0.1.0-py3-none-any.whl
- Upload date:
- Size: 57.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2082d409243c476e99e40e2c53136334cd28cd890a5bfd02d98ae9fab4a8c042
|
|
| MD5 |
a7bc2b288a0e3831fe16dbf4dbbdfb3f
|
|
| BLAKE2b-256 |
ce28f779081bb3e534d4580e7b74e82dee2532766d61f5f91ca5827bdf472fae
|
