sinapsis-vllm 0.2.0

Sinapsis templates for text, batch, streaming, and multimodal generation with vLLM.

Sinapsis vLLM

Sinapsis templates for text, batch, streaming, and multimodal generation with vLLM.

🐍 Installation • 🚀 Features • 📚 Usage example • 📙 Documentation • 🔍 License

The sinapsis-vllm package provides Sinapsis templates built on top of vLLM for high-throughput text generation, batch inference, streaming, and vision-language completion.

🐍 Installation

Install using your preferred package manager. We strongly recommend using uv.

Install the base package:

uv pip install sinapsis-vllm --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

pip install sinapsis-vllm --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] vLLMMultiModal requires the optional multimodal extra:

uv pip install sinapsis-vllm[multimodal] --extra-index-url https://pypi.sinapsis.tech

Or install all optional dependencies:

uv pip install sinapsis-vllm[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

• vLLMTextCompletion: Standard packet-native text generation with vLLM.
• vLLMBatchTextCompletion: Batched generation for multiple conversations in one execution pass.
• vLLMStreamingTextCompletion: Async streaming variant that yields partial packets during generation.
• vLLMMultiModal: Vision-language completion using image packets from container.images.

🧩 Common Attributes

• init_args (vLLMInitArgs, required): Engine configuration such as llm_model_name, tensor_parallel_size, dtype, quantization, trust_remote_code, max_model_len, and memory settings.
• completion_args (vLLMCompletionArgs, required): Request-time generation parameters such as temperature, top_p, top_k, min_p, max_tokens, penalties, and structured-output settings.
• cleanup_response (bool, optional): Whether to clean leaked transport or template markers from the final response.
• reasoning_start_tag / reasoning_end_tag (str | None, optional): Optional tags used to extract reasoning into the packet reasoning field and keep the final answer in response.

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis vLLM.

📚 Usage example

The following agent runs one vLLM text-completion step from an LLMConversationPacket.

Config

agent:
  name: my_test_agent

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name: LLMConversationInput
    class_name: LLMConversationInput
    template_input: InputTemplate
    attributes:
      prompt: "Give three short tips for staying organized during a busy week."
      system_prompt: "You are a helpful assistant."

  - template_name: vLLMTextCompletion
    class_name: vLLMTextCompletion
    template_input: LLMConversationInput
    attributes:
      init_args:
        llm_model_name: "Qwen/Qwen3-1.7B"
        max_model_len: 4096
        dtype: auto
        seed: 42
        gpu_memory_utilization: 0.9
        cpu_offload_gb: 0
        max_num_seqs: 8
        disable_log_stats: true
      completion_args:
        temperature: 0.7
        top_p: 0.8
        top_k: 20
        min_p: 0
        max_tokens: 2048
        seed: 42

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.