llama-index llms nvidia api catalog integration
Project description
NVIDIA's LLM connector
Install the connector,
pip install llama-index-llms-nvidia
With this connector, you'll be able to connect to and generate from compatible models available as hosted NVIDIA NIMs, such as:
- Google's gemma-7b
- Mistal AI's mistral-7b-instruct-v0.2
- And more!
First, get a free API key. Go to https://build.nvidia.com, select a model, click "Get API Key".
Store this key in your environment as NVIDIA_API_KEY
.
Then, try it out.
from llama_index.llms.nvidia import NVIDIA
from llama_index.core.llms import ChatMessage, MessageRole
llm = NVIDIA()
messages = [
ChatMessage(
role=MessageRole.SYSTEM, content=("You are a helpful assistant.")
),
ChatMessage(
role=MessageRole.USER,
content=("What are the most popular house pets in North America?"),
),
]
llm.chat(messages)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_llms_nvidia-0.1.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e82c4b43aa08be1d3e7e4fac5b5a0c6156f85c64c52a4bd2d559fb64618078c6 |
|
MD5 | c538900ce643f95a053a507a55a7487a |
|
BLAKE2b-256 | 12dea2816df77fc30806573dba80a3bb6ee29ac1012c1c2e3237c6b158037255 |
Close
Hashes for llama_index_llms_nvidia-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c73695945405d2f568a60b75d2a17a5d3b39be63152138e9686b01a4a66d4f8 |
|
MD5 | 1fcd69a7367860811fb31487714fe81c |
|
BLAKE2b-256 | 5d0dedcc6d8147c788f3cc48a029e3ef5294bd9579240f43df462d839af2e18f |