Bridge between Unsloth and Ollama

A utility for registering LoRA adapters into Ollama without weight fusion or GGUF quantization. It's tested on MLX models so far, but it should support other variants as well.

Standard conversion paths from Unsloth tuned model (mlx_vlm.fuse $\rightarrow$ convert_hf_to_gguf) are CPU-bound and result in multi-gigabyte files for a 150MB adapter change. This tool bridge MLX/Unsloth adapters to the HF PEFT schema, allowing for ADAPTER registration in Ollama. It removes the need for de-quantization and re-quantization cycles which prevents additional quantization degradation.

Installation

pip install lora-ollama-bridge

Or install from source:

git clone https://github.com/filtercodes/LoRA-Ollama-Bridge.git
cd LoRA-Ollama-Bridge
pip install -e .

NOTE - bridge support is currently pending upstream review in Ollama PR #17377. To use this tool right now, run the patched Ollama built from source:

git clone https://github.com/filtercodes/ollama.git
cd ollama
go build .
./ollama serve

Quickstart

1. Register Adapter in Ollama

Point the tool to the adapter directory (containing adapters.safetensors and adapter_config.json). It converts the tensors to HuggingFace PEFT schema and registers the model in Ollama:

lora-ollama-bridge -i ./mlx_adapters --name new-fine-tuned-model

2. Convert Adapter Only (Skip Ollama)

To convert the adapter format without calling the Ollama API:

lora-ollama-bridge -i ./mlx_adapters -o ./converted_adapter --skip-ollama

3. Custom Checkpoint & System Prompt

Specify a checkpoint file, system prompt, or context length:

lora-ollama-bridge \
  -i ./mlx_adapters \
  -w 0000400_adapters.safetensors \
  --name gemma4-fine-tune \
  -s "You are a helpful assistant." \
  --num-ctx 65536

4. Passing Modelfile Template

Alternatively use Modelfile directly rather than typing CLI flags:

lora-ollama-bridge -i ./mlx_adapters -f my_custom.modelfile --name gemma4-custom

CLI Reference

Requirements

• Python 3.10+
• Dependencies: mlx, torch, safetensors, requests, jinja2

Testing

python3 -m unittest discover -s tests