sinapsis-llama-cpp 0.5.1

Sinapsis templates for llama.cpp text generation.

Sinapsis LLaMA CPP

Sinapsis templates for local GGUF-backed text completion, streaming, and MCP with llama-cpp-python.

🐍 Installation • 🚀 Features • 📚 Usage example • 📙 Documentation • 🔍 License

The sinapsis-llama-cpp package provides Sinapsis templates built on top of llama-cpp-python for running local or Hugging Face-hosted GGUF models through LLMConversationPacket.

🐍 Installation

Install using your preferred package manager. We strongly recommend using uv.

Install the base package:

uv pip install sinapsis-llama-cpp --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

pip install sinapsis-llama-cpp --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] If you also want the upstream llama-cpp-python HTTP server, install the optional server extra:

uv pip install sinapsis-llama-cpp[server] --extra-index-url https://pypi.sinapsis.tech

Or install all optional dependencies:

uv pip install sinapsis-llama-cpp[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

• LLaMACPPTextCompletion: Standard llama.cpp chat completion using LLMConversationPacket.
• LLaMACPPStreamingTextCompletion: Async streaming variant that yields partial packets during generation.
• LLaMACPPTextCompletionWithMCP: llama.cpp chat completion with packet-native MCP tool state.

🧩 Common Attributes

• init_args (LLaMACPPInitArgs, required): llama.cpp runtime arguments.
  • llm_model_name (str, required): Hugging Face repo id or local directory containing the GGUF file.
  • llm_model_file (str, required): GGUF file name to load.
  • Additional runtime controls include n_ctx, n_threads, n_gpu_layers, flash_attn_type, tensor_split, use_mmap, use_mlock, seed, and chat_format.
• completion_args (LLaMACPPCompletionArgs, required): Request-time generation parameters such as max_tokens, temperature, top_p, top_k, min_p, penalties, stop sequences, and structured-output settings.
• reasoning_start_tag / reasoning_end_tag (str | None, optional): Tags used to extract reasoning into LLMConversationPacket.reasoning before the final response is cleaned.

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis LLaMA CPP.

📚 Usage example

The following agent runs one local llama.cpp text-completion step using LLMConversationInput.

Config

agent:
  name: text_completion
  description: Single-shot llama-cpp text completion for Q&A and text generation.

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name: LLMConversationInput
    class_name: LLMConversationInput
    template_input: InputTemplate
    attributes:
      prompt: Give three short tips for staying organized during a busy week.
      system_prompt: You are a helpful assistant.

  - template_name: LLaMACPPTextCompletion
    class_name: LLaMACPPTextCompletion
    template_input: LLMConversationInput
    attributes:
      init_args:
        llm_model_name: unsloth/Qwen3.5-9B-GGUF
        llm_model_file: Qwen3.5-9B-Q4_K_M.gguf
        n_ctx: 8192
        n_threads: 8
        n_gpu_layers: -1
        flash_attn_type: -1
        seed: 10
      completion_args:
        max_tokens: 4096
        temperature: 0.2
        seed: 10

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.