llm-smollm2

SmolLM2-135M-Instruct.Q4_1 for LLM. Background on this project.

Installation

Install this plugin in the same environment as LLM.

llm install llm-smollm2

If you have uv installed you can chat with the model without any installation step like this:

uvx --with llm-smollm2 llm chat -m SmolLM2

Usage

This plugin bundles a full copy of the SmolLM2-135M-Instruct.Q4_1 quantized version of the SmolLM2-135M-Instruct model by HuggingFaceTB.

Once installed, run the model like this:

llm -m SmolLM2 'Are dogs real?'

Or to chat with the model (keeping it resident in memory):

llm chat -m SmolLM2

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-smollm2
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

llm install -e '.[test]'

To run the tests:

python -m pytest