OVOS OpenAI Plugin

This package uses the OpenAI Chat Completions API to provide OpenAI-compatible plugins for OpenVoiceOS. It works with any server that exposes the OpenAI /chat/completions contract, such as OpenAI, ollama, llama.cpp, vLLM, or LocalAI. Point api_url at the server's /v1 base to use it.

This package provides:

The plugins use the agents framework of ovos-plugin-manager (AbstractAgentEngine). A persona names the chat engine ovos-chat-openai-plugin.

Install

pip install --pre ovos-openai-plugin

Persona Usage

To create your own persona using an OpenAI-compatible server, create a .json in ~/.config/ovos_persona/llm.json:

{
  "name": "My Local LLM",
  "solvers": [
    "ovos-chat-openai-plugin"
  ],
  "ovos-chat-openai-plugin": {
    "api_url": "https://llama.smartgic.io/v1",
    "key": "sk-xxxx",
    "system_prompt": "You are a helpful assistant who gives very short and factual answers in maximum twenty words and you don't use emojis",
    "model": "llama3.1:8b"
  }
}

The solvers key lists chat engine plugins. It also accepts solver plugins, which are deprecated.

An OpenAI-compatible endpoint is not OpenAI, and the parameters that matter differ per provider and per model. extra_params is a mapping merged into the request body last, so it can add a field the plugin has no setting for (reasoning_effort, thinking, top_k, repetition_penalty...) or override one it always sends (max_tokens, temperature, stop...). A key set to null removes the field from the request instead of sending it, for a provider that rejects it. The conversation itself (messages) cannot be replaced or removed this way; that key is ignored and logged. A value that is not a mapping is ignored and logged rather than taking the assistant down mid-conversation.

{
  "ovos-chat-openai-plugin": {
    "api_url": "https://llama.smartgic.io/v1",
    "model": "llama3.1:8b",
    "extra_params": {
      "reasoning_effort": "none",
      "top_k": 40,
      "stop": null
    }
  }
}

Say "Chat with {name_from_json}" to enable it. See the ovos-persona README for more details.

This plugin also provides a default "Remote LLama" demo persona, pointing to a public server hosted by @goldyfruit.

RAG memory

ovos-openai-rag-memory-plugin (PersonaServerRAGMemory) is a persona memory plugin. Before each turn, it searches a vector store on an ovos-persona-server and injects the retrieved chunks into the conversation context. The persona's chat engine then answers. This plugin composes with any chat backend instead of owning the chat round-trip.

Set it as the persona's memory_module:

{
  "name": "kb-assistant",
  "solvers": ["ovos-chat-openai-plugin"],
  "ovos-chat-openai-plugin": {
    "api_url": "http://localhost:8080/v1",
    "key": "sk-none",
    "model": "my-model",
    "allow_system_prompts": true
  },
  "memory_module": "ovos-openai-rag-memory-plugin",
  "ovos-openai-rag-memory-plugin": {
    "api_url": "http://localhost:8337/openai/v1",
    "vector_store_id": "vs_...",
    "inject_mode": "system",
    "retrieval": {"max_num_results": 5}
  }
}

ovos-chat-openai-plugin removes system messages unless allow_system_prompts is true. The default inject_mode adds the retrieved context as a system message, so without that setting the model never sees the context.

inject_mode selects how retrieved context enters the prompt: system (separate system message, default), system_prompt, developer, user, or tool (a synthetic search_knowledge_base tool-call result). Retrieval (max_num_results, min_score, query_mode) and context formatting are configurable. This feature requires an ovos-persona version that passes config to memory plugins. See docs/rag-memory.md for the full configuration reference, how to build the vector store this plugin searches (files + embeddings, local or hosted), and its failure modes.

Dialog Transformer

Rewrite text dynamically based on a persona, such as simplifying explanations or mimicking a tone.

Example:

• rewrite_prompt: "rewrite the text as if you were explaining it to a 5-year-old"
• Input: "Quantum mechanics is a branch of physics that describes the behavior of particles at the smallest scales."
• Output: "Quantum mechanics is like a special kind of science that helps us understand really tiny things."

To enable this plugin, add the following to your mycroft.conf:

"dialog_transformers": {
    "ovos-dialog-transformer-openai-plugin": {
        "api_url": "https://api.openai.com/v1",
        "key": "sk-xxxx",
        "model": "gpt-4o-mini",
        "system_prompt": "Your task is to rewrite text as if it was spoken by a different character",
        "rewrite_prompt": "rewrite the text as if you were explaining it to a 5-year-old"
    }
}

The actual dialog text is appended after rewrite_prompt in the request.

Direct Usage

from ovos_openai_plugin import OpenAIChatEngine
from ovos_plugin_manager.templates.agents import AgentMessage, MessageRole

bot = OpenAIChatEngine({
    "key": "sk-XXX",
    "model": "gpt-4o-mini",
    "system_prompt": "You are helpful, creative, clever, and very friendly",
})

# one-shot answer
print(bot.get_response("describe quantum mechanics in simple terms"))

# multi-turn chat
reply = bot.continue_chat([
    AgentMessage(MessageRole.USER, "what is the capital of France?")
])
print(reply.content)

# stream complete sentences (TTS friendly)
for sentence in bot.stream_sentences([AgentMessage(MessageRole.USER, "tell me a short story")]):
    print(sentence)

Translation and summarization:

from ovos_openai_plugin import OpenAITextTranslator, OpenAISummarizer

tx = OpenAITextTranslator({"key": "sk-XXX", "model": "gpt-4o-mini"})
print(tx.translate("hello world", target="es-es"))

summary = OpenAISummarizer({"key": "sk-XXX", "model": "gpt-4o-mini"})
print(summary.summarize("a very long document ..."))

Remote Persona / Proxies

You can run any persona behind an OpenAI-compatible server with ovos-persona-server. This moves the workload to a standalone server, either for performance or to keep API keys in a single safe place. Then configure this plugin to point at your persona server as if it were OpenAI.

Documentation

• Configuration reference
• Available plugins
• Persona integration
• RAG memory

Credits

Developed by TigreGótico for OpenVoiceOS.

This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101135429.