ocr_translate_ollama 0.3.0

Plugin to implement ollama (LLMs) based translations for ocr_translate

Plugin ocr_translate_ollama

This is a plugin for ocr_translate that implements translations through ollama using Large Language Models (LLM)s.

Usage

For versions of the server >=0.2

• Install this by running pip install ocr_translate_ollama
• Add ocr_translate_ollama to your INSTALLED_APPS in settings.py
• Run the server with AUTOCREATE_VALIDATED_MODELS once

For versions of the server >=0.6

• Install through the server plugin manager

For versions of the server >=0.7

• Install through the server plugin manager OR manually by running pip install ocr_translate-google

Ollama name prefixes

To avoid cluttering the ollama server (or having to set a dedicated one), the plugin will create new models from the original one (with a dedicated system prompt) adding the prefix oct_ollama_ to the model names.

This behavior can be changed by setting the environment variable OCT_OLLAMA_PREFIX to a different value.

NOTE: The final _ is added by the plugin, so it should not be included in the prefix.

IMPORTANT

Ollama needs to be installed separately and reachable from the server (check the link for instructions). The environment variable OCT_OLLAMA_ENDPOINT should be set to the endpoint of the ollama server (including the /api).

Example:

export OLLAMA_ENDPOINT=http://localhost:11434/api

Depending on the RAM available on your system (CPU/GPU), you also may need to tune the variables

• OLLAMA_MAX_LOADED_MODELS
• OLLAMA_NUM_PARALLEL

when running the server.

For more information, check the ollama FAQ