Review-Assist

CLI tool to assist with change requests for GitLab (MRs), requesting review and bootstrap the process with auto-review from a local LLM inference server (BYOM) you configure.

!!! warning

This tool is in its early stage of development, there will be breaking changes.

Installation

Using pipx

$ pipx install review-assist

Using uv

$ uv tool install review-assist

From source

Clone the repo, build and install the CLI tool:

$ git clone https://gitlab.com/rmenage/review-assist
$ cd review-assist
$ make clean
$ make local-install && make build && make build-binary && make install

The tool will be installed in ~/.local/bin/ra. You can run it by running ra from the command line.

Configuration

You can view the current configuration by running:

ra config

The first time you run that command, a default configuration will be created if none exists.

The default location for the configuration is ~/.config/ra/review-assist.cfg.

You can also pass the -c PATH or --config-file PATH option to override the default configuration with a custom config file.

Usage

!!! info

Note that the model used in the example below and the chosen context window size (64000)
should work on any Mac or Linux machine with 16GB unified memory (it was tested on a base Mac mini M4).

First, ensure you have installed and started a local LLM server, and the Hugging Face CLI (for downloading models). For example for headless LM Studio (lms):

$ curl -fsSL https://lmstudio.ai/install.sh | bash
Install completed at /Users/myuser/.lmstudio/llmster/0.0.20-1.
Installation finished successfully! llmster is ready to launch.
To start the daemon, run:

    lms daemon up

To add lms to your PATH, either restart your shell or run:
    export PATH="/Users/myuser/.lmstudio/bin:$PATH"

$ lms daemon up
$ brew install hf

We can now download a model and import it into lms:

$ hf download hf://unsloth/gemma-4-E4B-it-GGUF/gemma-4-E4B-it-UD-Q8_K_XL.gguf
$ lms import --symbolic-link ~/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/snapshots/bfc15c382204943c3a8fff0c750b94ae2364d7a3/gemma-4-E4B-it-UD-Q8_K_XL.gguf
✔ Choose categorization option Auto search Hugging Face (Recommended for models downloaded from Hugging Face)
Searching for the model on Hugging Face using the file name...
Found the following repositories on Hugging Face containing this file:
✔ Please select the correct one | None of the above
Please specify the user and repository manually.
✔ Who is the creator of the model? unsloth
✔ What is the model name? gemma-4-E4B-it-UD-Q8_K_XL

Verify that the model is recognised and load it in memory:

$ lms ls
You have 2 models, taking up 8.80 GB of disk space.

LLM                  PARAMS    ARCH      SIZE       DEVICE    
gemma-4-e4b-it-ud    7.5B      gemma4    8.71 GB    Local     

EMBEDDING                               PARAMS    ARCH          SIZE        DEVICE    
text-embedding-nomic-embed-text-v1.5              Nomic BERT    84.11 MB    Local     
$ lms load gemma-4-e4b-it-ud -c 64000

Then modify the config file at ~/.config/ra/review-assist.cfg to look like this:

[adapters]
local_llm.provider = lmstudio
local_llm.model = gemma-4-e4b-it-ud
local_llm.api_url = http://127.0.0.1:41343/v1
local_llm.api_key = 
git_server.base_url = https://gitlab.com/api/v4/projects
git_server.private_token = <your gitlab token>

where <your gitlab token> should be replaced with your real GitLab private token.

Note: Alternatively to filling in the git_server.private_token key, you can instead set an environment variable. Use RA_GL_TOKEN to that effect:

$ export RA_GL_TOKEN=<your gitlab token>

Next, to review a Gitlab merge request, pass the project name and the merge request IID to the command:

ra changes review --project my-namespace/my-project --change-request 42

Supported LLM backends and tested models

Those are the inference engine currently supported:

1. LM Studio (default)
2. OpenAI API generic backend

The default model is google/gemma-3-1b, as its small size should fit most computers. However, it's not going to provide many good insights.

Gemma 4 family of models and Qwen 3.6 family of models are much more useful in my experience. I've tested with the following models:

Using MLX format (Mac only, on a 48GB unified memory MacBook Pro M4Pro):

• lmstudio-community/gemma-4-e4b-it-mlx
• google/gemma-4-26b-a4b-qat
• mistralai/devstral-small-2-2512
• qwen3.6-35b-a3b

Using GGUF format:

• unsloth/gemma-4-e4b-it-ud (tested on a 16GB unified memory Mac Mini M4)
• unsloth/Qwen3.5-9B-GGUF (tested on a 48GB unified memory MacBook Pro M4Pro using llama.cpp)

Supported Git servers

1. GitLab

Cookbook

Install a local llama.cpp server, download a model and start the server

$ brew install llama.cpp
$ brew install hf
$ hf download hf://unsloth/Qwen3.5-9B-GGUF/Qwen3.5-9B-UD-Q4_K_XL.gguf
$ llama-server \
    -hf unsloth/Qwen3.5-9B-GGUF:UD-Q4_K_XL \
    --temp 0.6 \
    --top-p 0.95 \
    --top-k 20 \
    --min-p 0.00 \
    -ngl 99 \
    -c 131072 \
    -np 1 \
    -fa on \
    --alias "unsloth/Qwen3.5-9B-GGUF" \
    --port 4321 \
    --reasoning off 

!!! info

The models downloaded by `hf download` are stored in `~/.cache/huggingface/hub/`.

If `hf download` wasn't run, `llama-server` would have still downloaded the model from Hugging Face the first time
it runs, and store the model in the same location.
However, I think it's better to download the models separately as multiple inference servers (some not capable of auto-downloading) can share the same location.
Models files can be very large, so if switching between multiple engines, a shared pool of models is advantageous.

The config file at ~/.config/ra/review-assist.cfg should look like this:

[adapters]
local_llm.provider = openai
local_llm.model = unsloth/Qwen3.5-9B-GGUF
local_llm.api_url = http://127.0.0.1:4321
local_llm.api_key = nokey
git_server.base_url = https://gitlab.com/api/v4/projects
git_server.private_token = my-token

Troubleshooting

You can increase logging verbosity by passing -v one or more times. That will also show the config file and log file location in terminal output.

If passed three times (-vvv), it will also show the full stack trace when an error occurs, and the posted reviews will have a debug panel with LLM details.

The log file is located at ~/.local/state/ra/review-assist.log.

Configuration errors

!!! failure RAConfigError: Configuration error with source: ~/.config/ra/review-assist.cfg Config file permissions are incorrect (it should be read/write only for owner - chmod 600)

Because the configuration can hold private token for the git server API, It must not be readable by other users on the system. Problem can be fixed by changing the permissions:

$ chmod 600 ~/.config/ra/review-assist.cfg

The default configuration file (the example used here), created on first run has already the correct permissions. This error is more likely to happen when supplying a custom configuration file (with the -c parameter) and forgetting to change the permissions beforehand.

LMStudio errors

!!! failure

`lmstudio.LMStudioServerError: Chat response error: The number of tokens to keep from the initial prompt is greater than the context length. Try to load the model with a larger context length, or provide a shorter input`

The context is too small to fit all the tokens generated by the review. You can try increasing the context size to the maximum possible for a given model. E.g: for the default model, google/gemma-3-1b, the maximum context size is 32768, so make sure to set the context to that size:

$ lms load google/gemma-3-1b -c 32768

For qwen3.6-35b-a3b, the maximum is 262144, but I found acceptable and memory-saving to use half of that

$ lms load qwen3.6-35b-a3b -c 131072

!!! failure

`lmstudio.LMStudioServerError: Model get/load error: Model loading was stopped due to insufficient system resources. Continuing to load the model would likely overload your system and cause it to freeze. If you think this is incorrect, you can adjust the model loading guardrails in settings.`

In my experience, it indicates that the model I selected is too big to fit the current available memory. Try selecting a smaller model is a way of resolving that issue.

Known issues

With LM Studio (both the desktop app and the lms CLI tool), Gemma 4 models have recently started spilling thinking tokens into the output.

I have future plans to implement structured output which will allow me a finer control on the output and help out with this issue. More near term, I have started looking into the LMStudio Python SDK for parameters that can influence that issue, without success so far.

If you still want to use them, they work fine with llama.cpp with reasoning turned off:

$ llama-server \
    -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q8_K_XL \
    --temp 1 \
    --top-p 0.95 \
    --top-k 64 \
    --min-p 0.00 \
    -ngl 99 \
    -c 131072 \
    -np 1 \
    -fa on \
    --alias "unsloth/gemma-4-E4B-it-GGUF" \
    --port 4321 \
    --reasoning off

Contributing

See CONTRIBUTING.md