A collection of AI blueprints for Covalent Cloud.
Project description
Covalent Blueprints (AI)
A collection of AI blueprints for Covalent, built using the covalent-blueprints
library.
Installation
pip install -U covalent-blueprints-ai
Contents
This repository currently contains the following blueprints.
name | description |
---|---|
llama_chatbot |
Deploys a Llama-like chatbot service. |
sdxl_basic |
Deploys a stable-diffusion like image generator service. |
vllm_basic |
Deploys an LLM service using vLLM. |
lora_fine_tuning |
Fine-tunes and deploys a new LLM. |
Example
LoRA Fine-Tuning
from covalent_blueprints_ai import lora_fine_tuning
# Generate blueprint
bp = lora_fine_tuning(
model_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
num_train_epochs=2,
data="./imdb_sample.json", # local file
split="train"
)
# Set/modify compute resources
bp.executors.service_executor = cc.CloudExecutor(
gpu_type="v100",
num_cpus=6,
num_gpus=1,
time_limit="01:00:00",
memory="15GB"
)
# Run the blueprint's fine-tune and deploy workflow
fine_tuned_llm_client = bp.run()
print(fine_tuned_llm_client)
Output:
╭──────────────────────────────── Deployment Information ────────────────────────────────╮
│ Name LoRA Fine-Tuned LLM │
│ Description Serves a LoRA fine-tuned LLM for text generation. │
│ Function ID 66772d7fa1a393eb38302d77 │
│ Address https://fn.prod.covalent.xyz/066772d7fa1a393eb38302d77 │
│ Status ACTIVE │
│ Auth Enabled Yes │
╰────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────── Endpoints ───────────────────────────────────────╮
│ Route POST /generate │
│ Streaming No │
│ Description Generate text from a prompt. │
│ │
│ Route POST /stream │
│ Streaming No │
│ Description Generate text from a prompt and stream the response. │
╰────────────────────────────────────────────────────────────────────────────────────────╯
Authorization token:
pusCwZfDngoFVfJmNsEZWz8ZJUzPPnTGmkO7_GQH0OUkch6wK6sFpbCSm_a6aXjWKJmUV6ICXMCkbwYF_qMRYQ
Use the service client for generation:
response = fine_tuned_llm_client.generate(
prompt="The Lord of The Rings movies are an amazing trilogy! --"
)
print(response) # 0 - negative, 1 - positive
The Lord of The Rings movies are an amazing trilogy! -- 1
Tear down the service before its time limit:
fine_tuned_llm_client.teardown()
See examples/lora_fine_tuning.ipynb
for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for covalent_blueprints_ai-0.0.2rc6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 696a0968e66a82379e6c26a62be4e03a6a0cfb4d7326f5871b99647cb5e27aa3 |
|
MD5 | 0465fdf89853254f7dfd93f99c296f51 |
|
BLAKE2b-256 | 8ecf651d480205fb344ca642973c6d60ddf04491dec25a5ea981c89be9850275 |
Close
Hashes for covalent_blueprints_ai-0.0.2rc6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22a03c9c3558fc75730732bd06f54e6d83cb945f528789455e6f618fd30e18ed |
|
MD5 | 1656ab1397cbc33452a6e399d249ba2a |
|
BLAKE2b-256 | b4e626a704510204f7219cbebb654b84196e2b36f61dfbcb24c1f5a66f178472 |