llama-index llms nvidia api catalog integration
Project description
NVIDIA's LLM connector
Install the connector,
pip install llama-index-llms-nvidia
With this connector, you'll be able to connect to and generate from compatible models available as hosted NVIDIA NIMs, such as:
- Google's gemma-7b
- Mistal AI's mistral-7b-instruct-v0.2
- And more!
First, get a free API key. Go to https://build.nvidia.com, select a model, click "Get API Key".
Store this key in your environment as NVIDIA_API_KEY
.
Then, try it out.
from llama_index.llms.nvidia import NVIDIA
from llama_index.core.llms import ChatMessage, MessageRole
llm = NVIDIA()
messages = [
ChatMessage(
role=MessageRole.SYSTEM, content=("You are a helpful assistant.")
),
ChatMessage(
role=MessageRole.USER,
content=("What are the most popular house pets in North America?"),
),
]
llm.chat(messages)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_llms_nvidia-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 990fd1291cf36ce7962be1845d9e8b7eda90d9ad7f9614404424a67f95ee7f2b |
|
MD5 | 2bc7c99169bf62fce8208a17d5a1afd6 |
|
BLAKE2b-256 | 8e325590f5ecb77e91da97eba09e8912baec985c87720232233d7b1f3475c863 |
Close
Hashes for llama_index_llms_nvidia-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97b08bc9d474ecf3ac02bdde0b5165030f385f91f0b1b596e4f2ffb027d7c6d3 |
|
MD5 | c37e176386046640c0eba9c41f45957e |
|
BLAKE2b-256 | feaf67d7d508e0e91fc9bceb25aff6952c7164b8b95daa77a8bd6f0d461cc479 |