This package offers templates for building text-to-text, image-to-text, and tool-enabled conversational chatbots using Anthropic's Claude models.

• AnthropicTextGeneration: Template for text and code generation with Claude models using the Anthropic API.

• AnthropicMultiModal: Template for multimodal chat processing using Anthropic's Claude models.

• AnthropicWithMCP: Template for Claude chat workflows that expose LLMConversationPacket.tools and consume tool calls/results.

For specific instructions and further details, see the README.md.

This package provides the shared chatbot foundation for building and processing LLMConversationPacket flows.

• LLMConversationInput: Template for creating conversation packets from prompts, system prompts, and identity fields.

• QueryContextualizeFromFile: Template for attaching document context from preloaded generic_data entries.

• EndpointLLMCompletion: Template for connecting to OpenAI-compatible LLM endpoints such as OpenAI, Ollama, llama.cpp server, vLLM server, Gemini, and similar APIs.

• EndpointLLMCompletionWithMCP: Template for MCP-enabled chat workflows on top of OpenAI-compatible endpoints. Currently validated with llama.cpp served Qwen 3.5 models.

For specific instructions and further details, see the README.md.

This package provides local GGUF-backed inference templates built on top of llama-cpp-python.

• LLaMACPPTextCompletion: Local text completion with structured-output support.

• LLaMACPPStreamingTextCompletion: Streaming text completion for partial packet updates during generation.

• LLaMACPPTextCompletionWithMCP: Local text completion with packet-native MCP tool state.

For specific instructions and further details, see the README.md.

This package provides ingestion, retrieval, and reranking templates built on top of LlamaIndex.

• CodeEmbeddingNodeGenerator: Template to generate nodes for a code base.

• EmbeddingNodeGenerator: Template for generating text embeddings using the HuggingFace model.

• LLaMAIndexInsertNodes: Template for inserting embeddings (nodes) into PostgreSQL vector tables through PGVectorStore.

• LLaMAIndexNodeRetriever: Template for retrieving nodes and attaching them as packet contexts.

• LLaMAIndexReranker: Template for reranking selected retrieved contexts before a downstream LLM consumes them.

• LLaMAIndexSemanticCacheLookup: Reuse cached LLM responses from PGVector when a semantically similar prompt is found under the same request scope.

• LLaMAIndexSemanticCacheWrite: Persist completed LLM responses into a PGVector-backed semantic cache for future reuse.

For specific instructions and further details, see the README.md.

This package provides persistent memory functionality for Sinapsis agents using Mem0, supporting both managed (Mem0 platform) and self-hosted backends.

• Managed templates: Mem0ManagedAdd, Mem0ManagedGetAll, Mem0ManagedGetMemory, Mem0ManagedSearch, Mem0ManagedDeleteAll, Mem0ManagedDeleteMemory, Mem0ManagedReset
• OSS templates: Mem0OSSAdd, Mem0OSSGetAll, Mem0OSSGetMemory, Mem0OSSSearch, Mem0OSSDeleteAll, Mem0OSSDeleteMemory, Mem0OSSReset

For specific instructions and further details, see the README.md.

This package provides persistent chat history storage for Sinapsis agents using LLMConversationPacket workflows across SQL backends.

• ChatHistoryFetch: Template for retrieving stored chat history and attaching it to packet messages.

• ChatHistorySave: Template for persisting the current conversation turn from a packet.

• ChatHistoryDelete: Template for deleting stored chat history using explicit user_id / session_id scope and optional filters.

• ChatHistoryReset: Template for dropping and recreating the configured chat history table.

The package supports sqlite by default, with postgres and supabase available through the optional postgres extra.

For specific instructions and further details, see the README.md.

This package offers a suite of templates for running LLMs using vLLM, a high-throughput and memory-efficient inference engine for serving large language models.

• vLLMTextCompletion: Template for text completion using vLLM with support for structured outputs.

• vLLMBatchTextCompletion: Template for batched text completion using vLLM's continuous batching engine. Processes multiple conversations in a single batch for improved throughput.

• vLLMStreamingTextCompletion: Streaming version of vLLMTextCompletion for real-time response generation.

• vLLMMultiModal: Template for multimodal (text + image) completion using vLLM. Supports vision-language models like Qwen-VL.

For specific instructions and further details, see the README.md.

IMPORTANT: This Docker image depends on the sinapsis-nvidia:base image. For detailed instructions, please refer to the Sinapsis README.

1. Build the sinapsis-chatbots image:

docker compose -f docker/compose.yaml build

2. Start one webapp service

• llama-cpp chatbot:

docker compose -f docker/compose_apps.yaml up chatbot-llama-cpp -d

• vLLM chatbot:

docker compose -f docker/compose_apps.yaml up chatbot-vllm -d

• vLLM multimodal chatbot:

docker compose -f docker/compose_apps.yaml up chatbot-vllm-multimodal -d

• Anthropic chatbot:

export ANTHROPIC_API_KEY=your_api_key
docker compose -f docker/compose_apps.yaml up chatbot-anthropic -d

• Anthropic multimodal chatbot:

export ANTHROPIC_API_KEY=your_api_key
docker compose -f docker/compose_apps.yaml up chatbot-anthropic-multimodal -d

• Mem0 chatbot:

export MEM0_API_KEY=your_api_key
docker compose -f docker/compose_apps.yaml up chatbot-mem0 -d

• RAG chatbot:

docker compose -f docker/compose_apps.yaml up chatbot-rag -d

• Optional: vLLM chatbot using the separately served vLLM API:

docker compose -f docker/compose_vllm_server.yaml up -d
docker compose -f docker/compose_apps.yaml up chatbot-vllm-endpoint -d

The standalone vLLM server can be customized with HF_TOKEN, VLLM_MODEL, VLLM_SERVED_MODEL_NAME, VLLM_HOST, VLLM_PORT, VLLM_EXPOSED_PORT, VLLM_DTYPE, VLLM_MAX_MODEL_LEN, VLLM_MAX_NUM_SEQS, VLLM_GPU_MEMORY_UTILIZATION, VLLM_CPU_OFFLOAD_GB, and VLLM_SEED. Example:

VLLM_MODEL=Qwen/Qwen3.5-4B VLLM_EXPOSED_PORT=9001 VLLM_CPU_OFFLOAD_GB=8 \
docker compose -f docker/compose_vllm_server.yaml up -d

• Optional: llama.cpp chatbot using the separately served llama.cpp API:

docker compose -f docker/compose_llama_cpp_server.yaml up -d
docker compose -f docker/compose_apps.yaml up chatbot-llama-cpp-endpoint -d

The standalone llama.cpp server can be customized with HF_TOKEN, LLAMA_CPP_HOST, LLAMA_CPP_PORT, LLAMA_CPP_EXPOSED_PORT, LLAMA_CPP_HF_MODEL_REPO_ID, LLAMA_CPP_MODEL_FILE, LLAMA_CPP_MODEL_ALIAS, LLAMA_CPP_N_CTX, LLAMA_CPP_N_THREADS, LLAMA_CPP_N_GPU_LAYERS, and LLAMA_CPP_SEED.

3. Check the logs

• llama-cpp chatbot:

docker logs -f sinapsis-chatbot-llama-cpp

• vLLM chatbot:

docker logs -f sinapsis-chatbot-vllm

• vLLM multimodal chatbot:

docker logs -f sinapsis-chatbot-vllm-multimodal

• vLLM endpoint chatbot:

docker logs -f sinapsis-chatbot-vllm-endpoint

• llama.cpp endpoint chatbot:

docker logs -f sinapsis-chatbot-llama-cpp-endpoint

• Anthropic chatbot:

docker logs -f sinapsis-chatbot-anthropic

• Anthropic multimodal chatbot:

docker logs -f sinapsis-chatbot-anthropic-multimodal

• Mem0 chatbot:

docker logs -f sinapsis-chatbot-mem0

• RAG chatbot:

docker logs -f sinapsis-chatbot-rag

4. The logs will display the URL to access the webapp, e.g.:

Running on local URL:  http://127.0.0.1:7860

NOTE: The url may be different, check the output of logs.

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

To run a different variant with the generic chatbot entrypoint, update AGENT_CONFIG_PATH in the service environment to point to the desired YAML file under webapps/configs/.

To run the webapp using the uv package manager, follow these steps:

1. Sync the virtual environment:

uv sync --frozen

2. Install the workspace packages and webapp dependencies:

uv pip install sinapsis-chatbots[all] --extra-index-url https://pypi.sinapsis.tech

3. Run one webapp variant:

• llama-cpp chatbot:

uv run webapps/chatbot.py

• vLLM chatbot:

export AGENT_CONFIG_PATH=webapps/configs/vllm/vllm_text_generation.yaml
uv run webapps/chatbot.py

• vLLM multimodal chatbot:

export AGENT_CONFIG_PATH=webapps/configs/vllm/vllm_multimodal.yaml
uv run webapps/chatbot.py

• vLLM chatbot via separately served API:

docker compose -f docker/compose_vllm_server.yaml up -d
export AGENT_CONFIG_PATH=webapps/configs/served/vllm_server.yaml
uv run webapps/chatbot.py

• llama.cpp chatbot via separately served API:

docker compose -f docker/compose_llama_cpp_server.yaml up -d
export AGENT_CONFIG_PATH=webapps/configs/served/llama_cpp_server.yaml
uv run webapps/chatbot.py

• Anthropic chatbot:

export ANTHROPIC_API_KEY=your_api_key
export AGENT_CONFIG_PATH=webapps/configs/anthropic/anthropic_text_generation.yaml
uv run webapps/chatbot.py

• Anthropic multimodal chatbot:

export ANTHROPIC_API_KEY=your_api_key
export AGENT_CONFIG_PATH=webapps/configs/anthropic/anthropic_multimodal.yaml
uv run webapps/chatbot.py

• Mem0 chatbot:

export MEM0_API_KEY=your_api_key
uv run webapps/chatbot_with_mem0.py

• RAG chatbot:

uv run webapps/rag_chatbot.py

4. The terminal will display the URL to access the webapp, e.g.:

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.

To switch the generic chatbot entrypoint to a different provider or modality, change AGENT_CONFIG_PATH to the corresponding file under webapps/configs/.