llama-index llms nvidia api catalog integration
Project description
NVIDIA's LLM connector
Install the connector,
pip install llama-index-llms-nvidia
With this connector, you'll be able to connect to and generate from compatible models available as hosted NVIDIA NIMs, such as:
- Google's gemma-7b
- Mistal AI's mistral-7b-instruct-v0.2
- And more!
First, get a free API key. Go to https://build.nvidia.com, select a model, click "Get API Key".
Store this key in your environment as NVIDIA_API_KEY
.
Then, try it out.
from llama_index.llms.nvidia import NVIDIA
from llama_index.core.llms import ChatMessage, MessageRole
llm = NVIDIA()
messages = [
ChatMessage(
role=MessageRole.SYSTEM, content=("You are a helpful assistant.")
),
ChatMessage(
role=MessageRole.USER,
content=("What are the most popular house pets in North America?"),
),
]
llm.chat(messages)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_llms_nvidia-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e827fb2cf636d1174529196270b525bdf7faf63fee16b618fe544e288fe093b |
|
MD5 | 9ed7eb23f9c7b2771966d0f5fee0c109 |
|
BLAKE2b-256 | fcbdde57b622208167dd6c522d1b335a01a0e018eb85b62118563774605f73df |
Close
Hashes for llama_index_llms_nvidia-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8786025c03371e05c99181e2887335b451565a0ec86e2bff50febb1d2ba4183 |
|
MD5 | f45109c6ae1b86388eb9e0b674b80656 |
|
BLAKE2b-256 | 85c7bd301c95da4ba9cfb17d40a74053a0ca2ba86bc523045155423dae763c9d |