Sinapsis LLaMA CPP

Sinapsis templates for local GGUF-backed text completion, streaming, and MCP with llama-cpp-python.

🐍 Installation • 🚀 Features • 📚 Usage example • 📙 Documentation • 🔍 License

The sinapsis-llama-cpp package provides Sinapsis templates built on top of llama-cpp-python for running local or Hugging Face-hosted GGUF models through LLMConversationPacket.

🐍 Installation

Install using your preferred package manager. We strongly recommend using uv.

Install the base package:

uv pip install sinapsis-llama-cpp --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

pip install sinapsis-llama-cpp --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] If you also want the upstream llama-cpp-python HTTP server, install the optional server extra:
uv pip install sinapsis-llama-cpp[server] --extra-index-url https://pypi.sinapsis.tech
Or install all optional dependencies:
uv pip install sinapsis-llama-cpp[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

LLaMACPPTextCompletion: Standard llama.cpp chat completion using LLMConversationPacket.
LLaMACPPStreamingTextCompletion: Async streaming variant that yields partial packets during generation.
LLaMACPPTextCompletionWithMCP: llama.cpp chat completion with packet-native MCP tool state.

🧩 Common Attributes

init_args (LLaMACPPInitArgs, required): llama.cpp runtime arguments.
- llm_model_name (str, required): Hugging Face repo id or local directory containing the GGUF file.
- llm_model_file (str, required): GGUF file name to load.
- Additional runtime controls include n_ctx, n_threads, n_gpu_layers, flash_attn_type, tensor_split, use_mmap, use_mlock, seed, and chat_format.
completion_args (LLaMACPPCompletionArgs, required): Request-time generation parameters such as max_tokens, temperature, top_p, top_k, min_p, penalties, stop sequences, and structured-output settings.
reasoning_start_tag / reasoning_end_tag (str | None, optional): Tags used to extract reasoning into LLMConversationPacket.reasoning before the final response is cleaned.

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis LLaMA CPP.

📚 Usage example

The following agent runs one local llama.cpp text-completion step using LLMConversationInput.

Config

agent:
  name: text_completion
  description: Single-shot llama-cpp text completion for Q&A and text generation.

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name: LLMConversationInput
    class_name: LLMConversationInput
    template_input: InputTemplate
    attributes:
      prompt: Give three short tips for staying organized during a busy week.
      system_prompt: You are a helpful assistant.

  - template_name: LLaMACPPTextCompletion
    class_name: LLaMACPPTextCompletion
    template_input: LLMConversationInput
    attributes:
      init_args:
        llm_model_name: unsloth/Qwen3.5-9B-GGUF
        llm_model_file: Qwen3.5-9B-Q4_K_M.gguf
        n_ctx: 8192
        n_threads: 8
        n_gpu_layers: -1
        flash_attn_type: -1
        seed: 10
      completion_args:
        max_tokens: 4096
        temperature: 0.2
        seed: 10

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.

Metadata

Release files for sinapsis-llama-cpp 0.5.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sinapsis-llama-cpp 0.5.1
File	Size	Uploaded
sinapsis_llama_cpp-0.5.1.tar.gz	29.6 kB	Jun 11, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for sinapsis-llama-cpp 0.5.1
File	Interpreter	ABI	Platform	Reset
sinapsis_llama_cpp-0.5.1-py3-none-any.whl 32.4 kB Jun 11, 2026	Python 3	none	any	Details

Total release size: 62.0 kB

Release files / sinapsis_llama_cpp-0.5.1.tar.gz

Download URL	sinapsis_llama_cpp-0.5.1.tar.gz
Size	29.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`f5b282fc6cd364b838b6dde51705c84be345d53f4536773473855a31b4848b71`
BLAKE2b-256 checksum How to use checksums	`b0118830cd8224165b69fc53e9eda94bd02250a670be8cd62389e760f96ae918`
Upload date	Jun 11, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.5.16`

Release files / sinapsis_llama_cpp-0.5.1-py3-none-any.whl

Download URL	sinapsis_llama_cpp-0.5.1-py3-none-any.whl
Size	32.4 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`696d8b62da7cfae87d239406088819b1fb1fc5ae05b82cc32b7379bb0e117e95`
BLAKE2b-256 checksum How to use checksums	`fd6bbd2cc93d21dce6fc9b8de77960e0d64c71cd25e7cb07a91ad4fc92055299`
Upload date	Jun 11, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.5.16`