LLM plugin for Meta Llama2 on AWS Bedrock
Project description
llm-bedrock-meta
Plugin for LLM adding support for Meta LLama 2 and 3 models in Amazon Bedrock
Installation
Install this plugin in the same environment as LLM. From the current directory
llm install llm-bedrock-meta
Configuration
You will need to specify AWS Configuration with the normal boto3 and environment variables.
For example, to use the region us-west-2
and AWS credentials under the personal
profile, set the environment variables
export AWS_DEFAULT_REGION=us-west-2
export AWS_PROFILE=personal
Usage
This plugin adds models called bedrock-llama2-13b
, bedrock-llama2-70b
, bedrock-llama3-8b-instruct
, and bedrock-llama3-70b-instruct
. You can also use it with the aliases like bl2
or bl2-70
.
You can query them like this:
llm -m bl3-70-i "Ten great names for a new space station"
llm -m bl3-70-i "Ten great names for a new space station"
You can also chat with the model:
llm chat -m bl3-70-i
Options
-o max_gen_len 1024
, default 2_048: The maximum number of tokens to generate before stopping.-o verbose 1
, default 0: Output more verbose logging.-o temperature 0.8
, default 0.6: Use a lower value to decrease randomness in the response.top_p
, default 0.9: Use a lower value to ignore less probable options. Set to 0 or 1.0 to disable.
Use like this:
llm -m bl3-70-i -o max_gen_len 20 "Sing me the alphabet"
Ah, ah, ah! Here's the alphabet song for you:
A, B, C, D, E, F, G,
H, I, J, K, L, M, N, O,
P, Q, R, S, T, U, V, W,
X, Y, Z,
Now I know my ABCs,
Next time won't you sing with me?
Hope that brought a smile to your face!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llm_bedrock_meta-0.1.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 418b01a238accabfb6e258abb04e6bc7a0e8cc202904c474a06836f50af8fa54 |
|
MD5 | b5bfb4149dfc37a8e1df079055272df3 |
|
BLAKE2b-256 | 66aacbd2ac2c7e33bcc682e8a2cf6b2c08f2724468d5a6d1776c858b34cdebaa |