A collection of AI blueprints for Covalent Cloud.
Project description
Covalent Blueprints (AI)
A collection of AI blueprints for Covalent, built using the covalent-blueprints
library.
Installation
pip install -U covalent-blueprints-ai
Contents
This repository currently contains the following blueprints.
name | description |
---|---|
llama_chatbot |
Deploys a Llama-like chatbot service. |
sdxl_basic |
Deploys a stable-diffusion like image generator service. |
vllm_basic |
Deploys an LLM service using vLLM. |
lora_fine_tuning |
Fine-tunes and deploys a new LLM. |
Example
LoRA Fine-Tuning
from covalent_blueprints_ai import lora_fine_tuning
# Generate blueprint
bp = lora_fine_tuning(
model_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
num_train_epochs=2,
data="./imdb_sample.json", # local file
split="train"
)
# Set/modify compute resources
bp.executors.service_executor = cc.CloudExecutor(
gpu_type="v100",
num_cpus=6,
num_gpus=1,
time_limit="01:00:00",
memory="15GB"
)
# Run the blueprint's fine-tune and deploy workflow
fine_tuned_llm_client = bp.run()
print(fine_tuned_llm_client)
Output:
╭──────────────────────────────── Deployment Information ────────────────────────────────╮
│ Name LoRA Fine-Tuned LLM │
│ Description Serves a LoRA fine-tuned LLM for text generation. │
│ Function ID 66772d7fa1a393eb38302d77 │
│ Address https://fn.prod.covalent.xyz/066772d7fa1a393eb38302d77 │
│ Status ACTIVE │
│ Auth Enabled Yes │
╰────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────── Endpoints ───────────────────────────────────────╮
│ Route POST /generate │
│ Streaming No │
│ Description Generate text from a prompt. │
│ │
│ Route POST /stream │
│ Streaming No │
│ Description Generate text from a prompt and stream the response. │
╰────────────────────────────────────────────────────────────────────────────────────────╯
Authorization token:
pusCwZfDngoFVfJmNsEZWz8ZJUzPPnTGmkO7_GQH0OUkch6wK6sFpbCSm_a6aXjWKJmUV6ICXMCkbwYF_qMRYQ
Use the service client for generation:
response = fine_tuned_llm_client.generate(
prompt="The Lord of The Rings movies are an amazing trilogy! --"
)
print(response) # 0 - negative, 1 - positive
The Lord of The Rings movies are an amazing trilogy! -- 1
Tear down the service before its time limit:
fine_tuned_llm_client.teardown()
See examples/lora_fine_tuning.ipynb
for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for covalent_blueprints_ai-0.0.2rc8.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa13fb5ca90b611ae21bae5ec94c4435bdb44b5fa065597aed0a14fe319c87d1 |
|
MD5 | 0a96c34774f542a585f927e36b6eb25d |
|
BLAKE2b-256 | 29405be2a25f5fc7b6e9bbf92ab0cdb1081744220c9d2b098d8cabdcc86270ab |
Close
Hashes for covalent_blueprints_ai-0.0.2rc8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 34e58542210b8bfcf1f7cc098e1f85f18b35deddedc01d2eb5c8900086d4d9c8 |
|
MD5 | 9dd0ad6a8a18f6bbe72c49e1b08a6a85 |
|
BLAKE2b-256 | e48101494951e5dbcbb3cd3c9deccc0279221dbd294062d45deff4dfa9c4df4a |