llm-github-models 0.7

LLM plugin to access GitHub Models API

GitHub Models Plugin for LLM

This is a plugin for llm that uses GitHub Models via the Azure AI Inference SDK.

Installation

$ llm install llm-github-models

Usage

To set the API key, use the llm keys set github command or use the GITHUB_MODELS_KEY environment variable.

To get an API key, create a personal access token (PAT) inside GitHub Settings.

Learn about rate limits here

All model names are affixed with github/ to distinguish the OpenAI ones from the builtin models.

Example

$ llm prompt 'top facts about cheese' -m github/mistral-large                                                                                                                
Sure, here are some interesting facts about cheese:

1. There are over 2000 types of cheese: The variety of cheese is vast, with different flavors, textures, and aromas. This is due to factors like the type of milk used, the aging process, and the specific bacteria and mold cultures involved.

2. Cheese is an ancient food: The earliest evidence of cheese-making dates back to around 6000 BC, found in ancient Polish sites.

Image attachments

Multi-modal vision models can accept image attachments using the LLM attachments options:

llm -m github/Llama-3.2-11B-Vision-Instruct "Describe this image" -a https://static.simonwillison.net/static/2024/pelicans.jpg

Produces

This image depicts a dense gathering of pelicans, with the largest birds situated in the center, showcasing their light brown plumage and long, pointed beaks. The pelicans are standing on a rocky shoreline, with a serene body of water behind them, characterized by its pale blue hue and gentle ripples. In the background, a dark, rocky cliff rises, adding depth to the scene.

The overall atmosphere of the image exudes tranquility, with the pelicans seemingly engaging in a social gathering or feeding activity. The photograph's clarity and focus on the pelicans' behavior evoke a sense of observation and appreciation for the natural world.