NarDialPy

NarDialPy is a Python package for building and running narrative-driven, structured dialog systems — designed for social robots and conversational agents.

It lets you author complete conversations declaratively in JSON, then drive them from Python using voice, NLU, LLM, and browser-based screen services. The package handles session flow, branching logic, topic tracking, personalization, and screen output so you can focus on what the robot says and how conversations unfold.

---

Table of Contents

1. What is nardial?
2. Prerequisites & Setup
3. Providers & Initialization
4. Defining Dialogs in JSON
   • Dialog Structure
   • Dialog Types
     • Custom dialog types
   • Move Types
   • Key JSON Attributes
5. Demos / Creating a Session
6. Development

---

What is nardial?

The nardial package provides the building blocks for authoring and executing multi-turn conversations:

Component	Description
Dialog JSON	Conversations are written as structured JSON files. Each file holds one or more dialogs, each containing a sequence of moves that the robot performs.
Session Manager	Loads your dialog JSON, resolves a session agenda, and runs dialogs in order — checking dependencies and tracking state.
ConversationAgent	The runtime bridge to the hardware: it calls TTS, STT, LLM, and motion services on your chosen device.
Screen Provider	Optional browser-based display layer for transcripts, images, videos, HTML, buttons, and web input.
Dialog Logic	Checks eligibility rules (dependencies, variable requirements) before executing each dialog.

Typical use case:

• A social robot (Pepper, NAO, desktop agent) runs a structured conversation with a child or adult participant.
• The conversation is broken into named dialog blocks (greeting, story, chitchat, goodbye) authored in JSON.
• Python code wires up the device, credentials, and agenda — the JSON drives the actual content and branching.

Additional community demos are in the SIC Applications repository.

---

Prerequisites & Setup

1. Python IDE

Recommended: PyCharm or VS Code

2. Python

• Version: 3.12
• Download: https://www.python.org/downloads/release/python-3120/
• ⚠️ Ensure Python is added to your system PATH

3. In your project, create and activate your python virtual environment

# Windows
cd your_project_folder
python -m venv venv_myproject
venv_myproject\Scripts\activate

# macOS / Linux
cd your_project_folder
python -m venv venv_myproject
source venv_myproject/bin/activate

4. Install NarDial

Install the core package, then add extras for each service you intend to use:

pip install nardial

Extra	Enables	Install command
google-tts	Google Cloud Text-to-Speech	pip install "nardial[google-tts]"
elevenlabs	ElevenLabs Text-to-Speech	pip install "nardial[elevenlabs]"
dialogflow	Google Dialogflow NLU	pip install "nardial[dialogflow]"
openai	OpenAI GPT	pip install "nardial[openai]"
all	All of the above	pip install "nardial[all]"

For robot devices, install the matching SIC device extra directly:

pip install "social-interaction-cloud[alphamini]"   # Alphamini

Pepper and NAO are included in the base SIC package.

5. Configure Credentials

• Create a conf folder at the root of your project.
• Create a Google cloud project (for Dialogflow and google-tts services) and save the generated keyfile to conf/google/google_keyfile.json. Instructions here.
• Create an OpenAI account, and generate an API key and save it to conf/openai/.openai_env with the following content:

OPENAI_API_KEY="your key"

⚠️ Never commit credential files to version control.

6. Start Required Services

Run these in separate terminals before starting any demo:

# Windows
conf/redis/redis-server.exe conf/redis/redis.conf

# macOS / Linux
redis-server conf/redis/redis.conf

run-dialogflow
run-google-tts
run-gpt

---

Providers & Initialization

NarDialPy is built around a set of provider protocols. Each protocol defines a role (device, TTS, NLU, LLM, vector store) and multiple concrete implementations are available. You pick one implementation per role, instantiate it, and pass everything into ConversationAgent or SessionManager.

Available Providers

Role	Provider	Import path	Requires
Device	DesktopAdapter	nardial.providers.device.desktop	base
	PepperAdapter	nardial.providers.device.pepper	base
	NaoAdapter	nardial.providers.device.nao	base
	AlphaminiAdapter	nardial.providers.device.alphamini	social-interaction-cloud[alphamini]
	ReachyMiniAdapter	nardial.providers.reachy_mini	social-interaction-cloud[reachy-mini]
TTS	GoogleTTSProvider	nardial.providers.tts.google	nardial[google-tts]
	ElevenLabsTTSProvider	nardial.providers.tts.elevenlabs	nardial[elevenlabs]
	NaoqiTTSProvider	nardial.providers.tts.naoqi	base (uses device's built-in TTS)
	NullTTSProvider	nardial.providers.tts.null	base (prints to terminal)
NLU	DialogflowNLUProvider	nardial.providers.nlu.dialogflow	nardial[dialogflow]
	WrittenKeywordNLUProvider	nardial.providers.nlu.written_keyword	base (keyboard input)
LLM	OpenAIGPTProvider	nardial.providers.llm.openai_gpt	nardial[openai]
	EchoLLMProvider	nardial.providers.llm.echo	base (echoes user input)
Screen	ScreenProvider / SICScreenAdapter / PepperTabletScreenAdapter	nardial.providers.screen	browser display via SIC webserver
Vector store	RedisVectorStoreProvider	nardial.providers.vector_store.redis_store	base + running Redis
	NullVectorStoreProvider	nardial.providers.vector_store.null	base

---

Minimal setup (no external services)

Good for local development and testing — all I/O goes through the terminal:

import logging
from sic_framework.devices.desktop import Desktop

from nardial.providers.device.desktop import DesktopAdapter
from nardial.providers.tts.null import NullTTSProvider
from nardial.providers.nlu.written_keyword import WrittenKeywordNLUProvider
from nardial.conversation_agent import ConversationAgent

desktop = Desktop()
device = DesktopAdapter(desktop)
device.setup(logger=logging.getLogger())

agent = ConversationAgent(
    device=device,
    tts_provider=NullTTSProvider(),
    nlu_provider=WrittenKeywordNLUProvider(),
)

---

Desktop with cloud services

import json, logging
from sic_framework.devices.desktop import Desktop
from sic_framework.services.dialogflow.dialogflow import DialogflowConf

from nardial.providers.device.desktop import DesktopAdapter
from nardial.providers.tts.google import GoogleTTSProvider, GoogleTTSConf
from nardial.providers.tts.cacher import TTSCacher
from nardial.providers.nlu.dialogflow import DialogflowNLUProvider
from nardial.providers.llm.openai_gpt import OpenAIGPTProvider
from nardial.conversation_agent import ConversationAgent

desktop = Desktop()
device = DesktopAdapter(desktop)
device.setup(logger=logging.getLogger())

tts = GoogleTTSProvider(
    conf=GoogleTTSConf(speaking_rate=0.9, google_tts_voice_name="en-US-Neural2-F"),
    device=device,
    keyfile_path="conf/google/google_keyfile.json",
    tts_cacher=TTSCacher(tts_cache_dir="tts_cache"),
)

nlu = DialogflowNLUProvider(
    conf=DialogflowConf(keyfile_json=json.load(open("conf/google/google_keyfile.json"))),
    mic=desktop.mic,
)

llm = OpenAIGPTProvider(api_key="<YOUR_OPENAI_KEY>")

agent = ConversationAgent(
    device=device,
    tts_provider=tts,
    nlu_provider=nlu,
    llm_provider=llm,
)

---

Pepper robot

Swap the device adapter and TTS provider — everything else stays the same:

import logging
from sic_framework.devices import Pepper

from nardial.providers.device.pepper import PepperAdapter
from nardial.providers.tts.naoqi import NaoqiTTSProvider

pepper = Pepper(ip="<PEPPER_IP>")
device = PepperAdapter(pepper)
device.setup(logger=logging.getLogger())

tts = NaoqiTTSProvider(device=device, language="en")

Then pass device and tts to ConversationAgent as above.

---

Using SessionManager

SessionManager wraps ConversationAgent and adds dialog loading, eligibility checking, and session state:

from nardial.session_manager import SessionManager

manager = SessionManager(
    agent=agent,
    dialog_file="dialogs/my_dialogs.json",
    participant_id="user_42",
)
manager.run()

If a previous run crashed mid-session, pass resume=True to continue that same incomplete session (one whose ended_at is still None) instead of starting a new one — dialogs already completed in it are skipped:

manager = SessionManager(
    agent=agent,
    dialog_file="dialogs/my_dialogs.json",
    participant_id="user_42",
    resume=True,
)
manager.run()

See Developer Flow for more on session/state handling.

---

Defining Dialogs in JSON

All conversation content lives in a JSON file (or directory of JSON files). The file is a JSON array of dialog objects.

Dialog Structure

Every dialog has the following shared fields:

Field	Type	Required	Description
id	string	✅	Unique identifier referenced in session_agenda and dependencies
type	string	✅	Dialog type: "functional", "chitchat", "narrative", "llm_based", or a custom type
moves	array	✅	Ordered list of move objects the robot will perform
dependencies	array of strings		Dialog IDs that must have been completed before this dialog may run
variable_dependencies	array		Variables that must exist in the user model before this dialog may run
prerequisites	array		Actions run just before the dialog's first move: call a registered Python function, or set user-model variables (see below)

Example:

{
  "id": "greeting",
  "type": "functional",
  "functional_type": "greeting",
  "moves": [
    { "type": "say", "text": "Hi! I am your robot assistant." }
  ]
}

---

Dialog Types

Optional Top-Level characters

Dialogs can define reusable character voices:

{
  "characters": {
    "narrator": {
      "voice_settings": {
        "voice_id": "KTPVrSVAEUSJRClDzBw7",
        "language": "en"
      }
    }
  }
}

• characters is optional.
• Each key is a character name.
• Each value must contain a voice_settings object.
• voice_settings is validated at runtime against the active TTS provider.

Optional Top-Level prerequisites

Dialogs can list actions to run just before their first move, each time the dialog runs:

{
  "id": "multiplication",
  "type": "chitchat",
  "prerequisites": [
    { "execute": "choose_math_level", "args": [true, false] , "skip_dialog":  true},
    { "set_variable": { "math_left": 4, "math_right": 3 } }
  ],
  "moves": [
    { "type": "say", "text": "What is %math_left% times %math_right%?" }
  ]
}

Each entry does one of two things:

Key	Value	What happens
execute	name of a registered function	Calls that Python function. Optional args: a list is passed as separate arguments ([3, 4] calls f(3, 4)), any other value as a single argument, and without args the function is called with none.
set_variable	object	Merges the object into the user model, so its values can be used in %variable% placeholders and later dialogs.

In every prerequisite, "skip_dialog can be set to true of false, based on whether the dialog needs to be skipped if the prerequisite cannot be completed. "skip_dialog:true means that the dialog will be skipped if the prerequisite is not completed. If "skip_dialog: false is chosen, the dialog will still run, even if the prerequisite could not be completed. The default is false. Functions are registered in Python under the name used in execute, before the session runs:

from nardial.mini_dialogs import MiniDialog

@MiniDialog.register("choose_math_level")
def choose_math_level(last_correct, needed_help):
    if last_correct and not needed_help:
        level = 2
    else:
        level = 1
    return level

• Prerequisites run in the order listed, after the dialog has been picked. They can't change whether the dialog itself is eligible, only what happens once it runs.
• Registered functions must be regular (not async) functions. They get only the args from the JSON, not the dialog or the user model.
• If a function isn't registered or raises an error, the error is printed and the remaining prerequisites and the dialog still run.

---

1.functional

Utility dialogs for session management — greetings, farewells, and structural transitions.

Extra field	Type	Required	Description
functional_type	string	✅	"greeting" or "farewell"

{
  "id": "welcome",
  "type": "functional",
  "functional_type": "greeting",
  "moves": [
    { "type": "say", "text": "Hello! What is your name?" },
    {
      "type": "ask_open",
      "text": "Please tell me your name.",
      "set_variable": "first_name"
    }
  ]
}

---

2. chitchat

Short, theme-based conversations on everyday topics. Chitchat dialogs can be seeded with topics of interest so the system selects contextually relevant ones.

Extra field	Type	Required	Description
theme	string	✅	Broad category (e.g. "nature", "animals", "robots")
topics	array of strings		Specific keywords used to drive relevance matching

{
  "id": "favorite_animal",
  "type": "chitchat",
  "theme": "animals",
  "topics": ["animals", "pets"],
  "moves": [
    {
      "type": "ask_open",
      "text": "What is your favorite animal?",
      "set_variable": "favorite_animal",
      "add_interest_from_answer": true
    },
    { "type": "say", "text": "I like %favorite_animal% too!" }
  ]
}

---

3. narrative

Story-based dialogs that belong to a named thread and must be delivered in a specific order. Use position to sequence them and dependencies to enforce ordering.

Extra field	Type	Required	Description
thread	string	✅	Story thread name (e.g. "dreams") — groups related narrative dialogs
position	integer	✅	Order within the thread (1, 2, 3, …)

{
  "id": "dream_intro",
  "type": "narrative",
  "thread": "dreams",
  "position": 1,
  "moves": [
    { "type": "say", "text": "Sometimes I make up stories when I'm turned off." },
    {
      "type": "ask_yesno",
      "text": "Do you ever dream?",
      "set_variable": "dreams_yesno",
      "outcomes": { "yes": "dreams_yes" },
      "default_outcome": "dreams_no"
    },
    {
      "type": "branch",
      "on": "outcome",
      "cases": {
        "dreams_yes": [
          { "type": "say", "text": "That's great! Dreams are fascinating." }
        ],
        "dreams_no": [
          { "type": "say", "text": "No problem, I'll tell you about mine!" }
        ]
      }
    }
  ]
}

---

4. llm_based

A fully LLM-driven dialog where the robot and user engage in a free-form multi-turn exchange guided by a system prompt. No moves array is needed — the LLM generates all responses.

Extra field	Type	Required	Description
prompt	string	✅	System prompt guiding the LLM's behavior
max_turns	integer		Maximum back-and-forth turns (default: 5)
speak_first	boolean		If true (default), the robot speaks first; if false, it listens first
duration	number		Time limit in seconds for the whole exchange
quit_phrases	array of strings		User utterances that end the exchange early
quit_signal	string		Token the LLM can embed to signal it wants to end (default: "<<QUIT>>")
rag_enabled	boolean		Enable Retrieval-Augmented Generation (RAG). When true, the LLM retrieves context from a vector index before generating responses. Requires configuring a compatible RAG backend through social-interaction-cloud.
index_name	string	Required when rag_enabled is true	Name of the RAG index to query

{
  "id": "free_chat",
  "type": "llm_based",
  "prompt": "You are a friendly robot. Chat warmly with the child about their day. Ask follow-up questions. End with <<QUIT>> when the topic is exhausted.",
  "max_turns": 6,
  "speak_first": true,
  "quit_signal": "<<QUIT>>",
  "quit_phrases": ["goodbye", "stop", "done"],
  "moves": []
}

Custom dialog types

You can define your own dialog type in JSON. A custom type is based on an existing type and changes the rules that decide when its dialogs may run.

Type definitions go in their own JSON file (or a directory of them), separate from your dialogs. Mixing them is an error: a type definition in a dialog file, or a dialog in a types file, is reported and skipped. Pass the types file to SessionManager as dialog_types_path. It is loaded before the dialogs, so dialogs can use the new types:

manager = SessionManager(
    session_agenda=[...],
    agent=agent,
    dialog_types_path="dialog_types/my_types.json",
    dialog_json_path="dialog_json/my_dialogs.json",
)

Without SessionManager, call load_dialog_types(path) (from nardial.authoring) before loading the dialogs. Within the types file(s), the order of definitions doesn't matter, even when one type extends another.

A type definition looks like this:

{
  "define_type": "repeatable_chitchat",
  "extends": "chitchat",
  "description": "Chitchat that may come back in later sessions, but runs at most once per session.",
  "remove_rules": ["exclude_if_seen"],
  "add_rules": [{ "rule": "exclude_if_seen", "scope": "session" }]
}

Field	Type	Required	Description
define_type	string	✅	Name of the new type, used as type in dialogs
extends	string	✅	Type it is based on: a built-in type or another custom type
description	string		Free-text description
remove_rules	array of strings		Names of the base type's rules to drop
add_rules	array		Rules to add: a rule name, or an object with "rule" plus that rule's settings

Dialogs of the new type use the same fields as dialogs of the type it extends:

{
  "id": "music_lately",
  "type": "repeatable_chitchat",
  "topics": ["music"],
  "moves": [{ "type": "say", "text": "What music have you been listening to lately?" }]
}

They are also picked by the same agenda slots. For example, a chitchat_slot can choose a repeatable_chitchat dialog.

Built-in types and their rules:

Type	Rules
functional	dependency_met
narrative	exclude_if_seen, dependency_met, variable_dependency_met, narrative_ordering
chitchat	exclude_if_seen, dependency_met, variable_dependency_met
llm_based	exclude_if_seen, dependency_met, variable_dependency_met

Available rules. A dialog may run only when all of its type's rules allow it.

Rule	Settings	Allows the dialog to run when
exclude_if_seen	scope: "participant" (default) or "session"	it hasn't been completed before (by this participant, or in this session)
dependency_met		every dialog in its dependencies has been completed
variable_dependency_met		every required variable in its variable_dependencies is set in the user model
narrative_ordering		every earlier position in its thread has been completed

Your own rules. Any other rule is written in Python and registered under a name, which add_rules can then use. Settings from the JSON are passed to the constructor:

from nardial.eligibility import EligibilityRule, register_rule

@register_rule("from_session")
class FromSessionRule(EligibilityRule):
    def __init__(self, min):
        self.min = min

    def is_eligible(self, dialog, context):
        # context.session_index: session number starting at 1, or None when unknown
        return context.session_index is not None and context.session_index >= self.min

{ "define_type": "returning_chitchat", "extends": "chitchat", "add_rules": [{ "rule": "from_session", "min": 2 }] }

Import the module that registers the rule before loading your dialogs. A rule can read context.user_model, context.completed_ids, context.session_completed_ids, context.session_index and context.registry. See docs/DEVELOPER_README.md for more, including defining types directly in Python.

---

Move Types

Moves are the individual steps inside a dialog's moves array. The robot executes them in order.

---

say

Speaks a piece of text. Variable placeholders (%variable_name%) are replaced at runtime.

Field	Type	Required	Description
type	string	✅	"say"
text	string	✅	Text to speak. Use %var% to insert stored variables.
character	string		Optional character name from top-level characters; if omitted, default provider voice settings are used.

{ "type": "say", "text": "Nice to meet you, %first_name%!" }

---

say_options

Speaks one randomly chosen response from a predefined list.

Field	Type	Required	Description
type	string	✅	"say_options"
options	array of strings	✅	Candidate utterances to choose from at runtime
character	string		Optional character name from top-level characters

{ "type": "say_options", "options": ["Hi!", "Hello!", "Hey there!"] }

---

ask_open

Asks a free-text question and listens for any spoken reply. The answer can be stored in a variable and used to drive branching.

Field	Type	Required	Description
type	string	✅	"ask_open"
text	string	✅	The question to ask
set_variable	string		Stores the extracted answer in the user model under this name
outcomes	object		Maps answer values to outcome labels. Use "*" as a wildcard for any non-empty answer.
default_outcome	string		Outcome label when no answer or no match is found
add_interest_from_answer	boolean		If true, adds the answer to the user's topics of interest
llm_followup	string		System prompt for an LLM-generated follow-up sentence after the user replies
llm_extract_entity	boolean		If true, the LLM extracts the key entity from the answer (e.g. "dog" from "my favorite animal is a dog") before it is stored; unrelated answers store nothing. Default false

{
  "type": "ask_open",
  "text": "What is your favorite color?",
  "set_variable": "favorite_color",
  "add_interest_from_answer": true
}

---

ask_yesno

Asks a yes/no question. The detected intent ("yes", "no", "dontknow") drives branching.

Field	Type	Required	Description
type	string	✅	"ask_yesno"
text	string	✅	The yes/no question to ask
set_variable	string		Stores the answer in the user model
outcomes	object		Maps "yes" / "no" / "dontknow" to outcome labels
default_outcome	string		Outcome label used as fallback
add_interest	string		Topic added to interest list when the user answers "yes"
llm_followup	string		System prompt for an LLM-generated follow-up sentence

{
  "type": "ask_yesno",
  "text": "Do you like dogs?",
  "set_variable": "likes_dogs",
  "add_interest": "dogs",
  "outcomes": { "yes": "likes_dogs_yes" },
  "default_outcome": "likes_dogs_no"
}

---

ask_options

Presents a multiple-choice question. The selected option drives branching and can be stored as a variable.

Field	Type	Required	Description
type	string	✅	"ask_options"
text	string	✅	The question to ask
options	array of strings	✅	The choices presented to the user
set_variable	string		Stores the selected option in the user model
outcomes	object		Maps option values to outcome labels
default_outcome	string		Outcome label when nothing matches
add_interest_from_variable	string		After storing, adds the named variable's value as a topic of interest
llm_followup	string		System prompt for an LLM-generated follow-up sentence

{
  "type": "ask_options",
  "text": "Which activity do you prefer — reading, walking, or cooking?",
  "options": ["reading", "walking", "cooking"],
  "set_variable": "preferred_activity",
  "outcomes": {
    "reading": "chose_reading",
    "walking": "chose_walking",
    "cooking": "chose_cooking"
  },
  "default_outcome": "chose_other"
}

---

ask_llm

Starts a multi-turn LLM-driven exchange within an otherwise scripted dialog. Useful for a single free-form segment inside a structured conversation.

Field	Type	Required	Description
type	string	✅	"ask_llm"
prompt	string	✅	System prompt for the LLM
max_turns	integer		Maximum turns (default: 5)
set_variable	string		Stores the user's last reply in the user model
quit_phrases	array of strings		User phrases that end the exchange early
quit_signal	string		Token the LLM emits to signal it wants to stop

{
  "type": "ask_llm",
  "prompt": "Ask one follow-up question to help the user commit to their plan for today. Keep it short and friendly.",
  "max_turns": 1,
  "set_variable": "plan_commitment"
}

---

branch

Selects and executes a list of sub-moves based on the current outcome or the value of a user model variable.

Field	Type	Required	Description
type	string	✅	"branch"
on	string	✅	"outcome" to branch on the last question's result, "variables" to branch based on whether to variabels in the user model have the same value, or a variable name to branch on its stored value
cases	object	✅	Maps condition values to arrays of sub-moves
variables	list		In case of branching on "variables", a list of the variables that need to be compared

{
  "type": "branch",
  "on": "outcome",
  "cases": {
    "chose_reading": [
      { "type": "say", "text": "Reading is a wonderful way to relax." }
    ],
    "chose_walking": [
      { "type": "say", "text": "A walk sounds refreshing!" }
    ]
  }
}

Branching on a stored variable (e.g. to react to an answer from an earlier dialog):

{
  "type": "branch",
  "on": "energy_level",
  "cases": {
    "high": [{ "type": "say", "text": "Start with a longer session." }],
    "low":  [{ "type": "say", "text": "Begin with just 10 calm minutes." }]
  }
}

Branching on comparison of two stored variables (e.g. to check whether a given answer is correct). Chooses branch "true" if the values of the variables are the same, branch "false" if not.

{
  "type": "branch",
  "on": "variables",
  "variables": ["given_answer", "correct_answer"],
  "cases": {
    "true": [{ "type":  "say", "text":  "That's correct!"}],
    "false": [{ "type":  "say", "text":  "That's incorrect. Let's try again."}]
  }
}

---

play

Plays an audio file through the device's speakers.

Field	Type	Required	Description
type	string	✅	"play"
audio	string	✅	Path to the audio file (.wav or .mp3)

{ "type": "play", "audio": "audio/chime.wav" }

---

motion_sequence

Plays a predefined motion sequence on the robot (Pepper / NAO).

Field	Type	Required	Description
type	string	✅	"motion_sequence"
motion_sequence	string	✅	Path or name of the motion sequence file

{ "type": "motion_sequence", "motion_sequence": "motions/Stand/Emotions/Positive/Happy_1" }

---

animation

Triggers a named animation behavior on the robot.

Field	Type	Required	Description
type	string	✅	"animation"
animation_name	string	✅	Name of the animation to play

{ "type": "animation", "animation_name": "animations/Stand/Gestures/Enthusiastic_4" }

---

timed_wait

Pause execution for a fixed duration before proceeding to the next move. Useful to add natural pauses or give the user time to look at the screen.

Field	Type	Required	Description
type	string	✅	"timed_wait"
duration_seconds	number	✅	Seconds to wait. (Alias: duration is also accepted.)

{ "type": "timed_wait", "duration_seconds": 2.5 }

---

wait_for_web_input

Suspend dialog execution until a matching web input event arrives or until a timeout elapses. The web UI can present buttons or a short input and emit web_input events which this move listens for.

Field	Type	Required	Description
type	string	✅	"wait_for_web_input"
prompt	string		Hint text shown in the web UI (not spoken)
options	array of strings		Accepted value strings from the web event (if omitted, any value is accepted)
timeout	number		Seconds to wait before falling back (omit for indefinite wait)
outcomes	object		Maps option value → outcome label for branching
default_outcome	string		Outcome used on timeout or when no event matches
set_variable	string		Stores the user's choice in the user model

{
  "type": "wait_for_web_input",
  "prompt": "Choose a sticker to show",
  "options": ["smile","thumbs","surprised"],
  "timeout": 15,
  "outcomes": {"smile": "picked_smile"},
  "default_outcome": "no_choice",
  "set_variable": "sticker"
}

---

show_image

Display an image on the connected screen (browser / tablet). Accepts a local path relative to the static assets directory or a full URL.

Field	Type	Required	Description
type	string	✅	"show_image"
src	string	✅	Path or URL to the image (aliases: image, path)

{ "type": "show_image", "src": "assets/images/robot_pet.png" }

---

show_video

Display a video on the screen. Can accept a local file path or an embeddable URL (e.g. a YouTube embed link).

Field	Type	Required	Description
type	string	✅	"show_video"
src	string	✅	Video file path or embeddable URL (aliases: video, path)

{ "type": "show_video", "src": "https://www.youtube.com/embed/VIDEO_ID" }

---

show_iframe

Embed an external web page inside an iframe on the screen. Use this for interactive web content served from a trusted origin.

Field	Type	Required	Description
type	string	✅	"show_iframe"
url	string	✅	URL to embed (aliases: src, iframe_url)

{ "type": "show_iframe", "url": "https://example.com/mini-game" }

---

show_html

Render a raw HTML snippet directly into the display area. The frontend inserts this via innerHTML — treat content as trusted and avoid injecting untrusted user data.

Field	Type	Required	Description
type	string	✅	"show_html"
html	string	✅	HTML snippet to render

{ "type": "show_html", "html": "<div class=\"card\">Hi there!</div>" }

---

black_screen

Clear the display (show a blank/black screen). No parameters — the next display move will restore content.

Field	Type	Required	Description
type	string	✅	"black_screen"

{ "type": "black_screen" }

---

keyboard_input

Display keyboard input on screen. Suspend dialog execution until keyboard input is given or until a timeout elapses.

Field	Type	Required	Description
type	string	✅	"keyboard_input"
prompt	string		Hint text shown in the web UI (not spoken)
timeout	number		Seconds to wait before falling back (omit for indefinite wait)
outcomes	object		Maps option value → outcome label for branching
default_outcome	string		Outcome used on timeout or when no event matches
set_variable	string		Stores the user's choice in the user model

{
  "type": "keyboard_input",
  "prompt": "How old are you?",
  "timeout": 15,
  "default_outcome": "no_age_given",
  "set_variable": "age"
}

---

go_to_dialog

Move to a specified new minidialog.

Field	Type	Required	Description
type	string	✅	"go_to_dialog"
dialog_id	string	✅	ID of the dialog to go to

{
  "type": "go_to_dialog",
  "dialog_id": "favorite_animal"
}

---

Key JSON Attributes

Attribute	Where used	Description

---

Demos / Creating a Session

All you need is a minimal Python script that wires up the device, loads the dialog JSON, and runs the session. You can follow the included demos to get started quickly.

Ready-to-run demos are included in the examples/ directory:

• Demo 1 — General Conversation (demo_general_conversation.py): A simple four-step conversation using a mix of narrative and functional dialogs
• Demo 2 — Structured Conversation (demo_structured_conversation.py): A more complete example that demonstrates all dialog types and move types, including ask_llm, play, motion_sequence, and animation
• Demo 3 — Screen Display (demo_screen_provider.py): Shows the browser-based screen UI with transcripts, images, iframes, HTML snippets, buttons, and text input
• Demo 4 — Pepper Tablet (demo_pepper_tablet.py): Uses the same screen UI on Pepper's tablet through the SIC webserver
• Demo 5 — Agenda System (demo_agenda_system.py): Uses every agenda-item type (narrative_slot, chitchat_slot, functional_slot, llm_dialog_ref, plain dialog ids) so SessionManager picks the next dialog dynamically instead of following a fixed list — see the Agenda System Guide for a plain-language walkthrough
• Demo 6 — Custom Dialog Type (demo_custom_dialog_type.py): Writes a custom eligibility rule in Python and defines two new dialog types in JSON (see Custom dialog types): a chitchat that comes back every session, and one that only runs for participants who said they have a pet

You can find additional demos in the SIC Applications repository

---

Development

For an architecture-level explanation of how JSON dialogs become running conversations, see Developer Flow. If you're building an application and want to understand dynamic session agendas rather than runtime internals, see the Agenda System Guide instead.

Run tests from the repository root:

python -m pytest -q