LLM plugin for models hosted by SambaNova
Project description
llm-sambanova
LLM plugin for models hosted by SambaNova
Installation
First, install the LLM command-line utility.
Now install this plugin in the same environment as LLM.
llm install llm-sambanova
Configuration
You will need an API key from SambaNova. You can obtain one here.
You can set that as an environment variable called OPENROUTER_KEY
, or add it to the llm
set of saved keys using:
llm keys set sambanova
Enter key: <paste key here>
Usage
To list available models, run:
llm models list
You should see a list that looks something like this:
SambaNova: sambanova/Meta-Llama-3.2-1B-Instruct
SambaNova: sambanova/Meta-Llama-3.2-3B-Instruct
SambaNova: sambanova/Meta-Llama-3.1-8B-Instruct
SambaNova: sambanova/Meta-Llama-3.1-8B-Instruct-8k
SambaNova: sambanova/Meta-Llama-3.1-70B-Instruct
SambaNova: sambanova/Meta-Llama-3.1-70B-Instruct-8k
SambaNova: sambanova/Meta-Llama-3.1-405B-Instruct
SambaNova: sambanova/Meta-Llama-3.1-405B-Instruct-8k
SambaNova: sambanovacompletion/Meta-Llama-3.2-1B-Instruct
SambaNova: sambanovacompletion/Meta-Llama-3.2-3B-Instruct
SambaNova: sambanovacompletion/Meta-Llama-3.1-8B-Instruct
SambaNova: sambanovacompletion/Meta-Llama-3.1-8B-Instruct-8k
SambaNova: sambanovacompletion/Meta-Llama-3.1-70B-Instruct
SambaNova: sambanovacompletion/Meta-Llama-3.1-70B-Instruct-8k
SambaNova: sambanovacompletion/Meta-Llama-3.1-405B-Instruct
SambaNova: sambanovacompletion/Meta-Llama-3.1-405B-Instruct-8k
...
To run a prompt against a model, pass its full model ID to the -m
option, like this:
llm -m sambanova/Meta-Llama-3.2-1B-Instruct "Five spooky names for a pet tarantula"
You can set a shorter alias for a model using the llm aliases
command like so:
llm aliases set 405 sambanova/Meta-Llama-3.1-405B-Instruct
Now you can prompt Claude using:
cat llm_sambanova.py | llm -m 405 -s 'write some pytest tests for this'
Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
cd llm-sambanova
python3 -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
pip install -e '.[test]'
To run the tests:
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llm_sambanova-0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0c1c436d72a7cdb6a29eef5d448b627391bb39bac2cab0ab3f4cfab23db66651 |
|
MD5 | 9cc8e0c63c7237a03595971d657107ae |
|
BLAKE2b-256 | 179a4d8d52e4fde38998b0c061fab146e81dd049f2fe643b546d6a5792ed3460 |