llama-index-llms-nvidia 0.1.0

llama-index llms nvidia api catalog integration

NVIDIA's LLM connector

Install the connector,

pip install llama-index-llms-nvidia

With this connector, you'll be able to connect to and generate from compatible models available as hosted NVIDIA NIMs, such as:

• Google's gemma-7b
• Mistal AI's mistral-7b-instruct-v0.2
• And more!

First, get a free API key. Go to https://build.nvidia.com, select a model, click "Get API Key". Store this key in your environment as NVIDIA_API_KEY.

Then, try it out.

from llama_index.llms.nvidia import NVIDIA
from llama_index.core.llms import ChatMessage, MessageRole

llm = NVIDIA()

messages = [
    ChatMessage(
        role=MessageRole.SYSTEM, content=("You are a helpful assistant.")
    ),
    ChatMessage(
        role=MessageRole.USER,
        content=("What are the most popular house pets in North America?"),
    ),
]

llm.chat(messages)