llm-bedrock 0.2

Run prompts against models hosted on AWS Bedrock

llm-bedrock

Run prompts against models hosted on AWS Bedrock

Installation

Install this plugin in the same environment as LLM.

llm install llm-bedrock

You'll need an access key and a secret key to use this plugin, with permission granted to access the Bedrock models. These step by step instructions can help you obtain those credentials.

Combine those into a access_key:secret_key format (joined by a colon) and paste that into:

llm keys set bedrock
# paste access_key:secret_key here

Usage

Run llm models to see the list of models. The Amazon Nova models are:

• us.amazon.nova-micro-v1:0 (alias: nova-micro) - cheapest and fastest, text only
• us.amazon.nova-lite-v1:0 (alias: nova-lite) - can handle text, images and PDFs
• us.amazon.nova-pro-v1:0 (alias: nova-pro) - can handle text, images and PDFs, best and most expensive

Run a prompt like this:

llm -m nova-pro 'a happy poem about a pelican with a secret'

Images and PDFs can be provided using the -a option, which takes a file path or a URL:

llm -m nova-lite 'describe this image' -a https://static.simonwillison.net/static/2024/pelicans.jpg

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-bedrock
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

llm install -e '.[test]'

To run the tests:

python -m pytest

To regenerate the captured HTTP responses:

PYTEST_BEDROCK_API_KEY="$(llm keys get bedrock)" python -m pytest --record-mode all