llama-index llms lmstudio integration
Project description
LlamaIndex Llms Integration: Lmstudio
pip install llama-index-llms-lmstudio
Usage Steps
- Open LM Studio App and go to the Local Server Tab
- In the Configuration settings, enable Apply Prompt Formatting
- Load the model of your choice
- Start your server
from llama_index.llms.lmstudio import LMStudio
llm = LMStudio(
model_name="Hermes-2-Pro-Llama-3-8B",
base_url="http://localhost:1234/v1",
temperature=0.7,
)
messages = [
ChatMessage(
role=MessageRole.SYSTEM,
content="You an expert AI assistant. Help User with their queries.",
),
ChatMessage(
role=MessageRole.USER,
content="What is the significance of the number 42?",
),
]
response = llm.chat(messages=messages)
print(str(response))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_llms_lmstudio-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d2d0157b23f4d5de7a335b093613240556df991ee93944a9ffdad9bcaabc8d7 |
|
MD5 | 9b4b81c23f107948c530b0014912fa4c |
|
BLAKE2b-256 | 1727f24397d5d196d79b59f43902ee5639b53adc2039666d6cd022eb72b24eb6 |
Close
Hashes for llama_index_llms_lmstudio-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4c85f4a9b687458f1a03b68809424469dee680996bfa2d2b767153df641af95 |
|
MD5 | d8335a7c8b34d4415408f97d63c6149e |
|
BLAKE2b-256 | b848ea2cd51fd7dc142f3be7c46f3a0504bbb4b40b0abc10f2114014378da225 |