Metadata-Version: 2.4
Name: nardial
Version: 0.4.2
Summary: Nardial Dialog System
Author-email: Mike Ligthart <m.e.u.ligthart@vu.nl>
Maintainer: Clara Fayyad, Landon Brown, Evelyne Prodescu, Eva Biesot
License-Expression: AGPL-3.0-only
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: social-interaction-cloud
Requires-Dist: pynput>=1.7
Provides-Extra: google-tts
Requires-Dist: social-interaction-cloud[google-tts]; extra == "google-tts"
Provides-Extra: elevenlabs
Requires-Dist: social-interaction-cloud[elevenlabs-tts]; extra == "elevenlabs"
Provides-Extra: dialogflow
Requires-Dist: social-interaction-cloud[dialogflow]; extra == "dialogflow"
Provides-Extra: openai
Requires-Dist: social-interaction-cloud[openai-gpt]; extra == "openai"
Provides-Extra: webserver
Requires-Dist: social-interaction-cloud[webserver]; extra == "webserver"
Provides-Extra: all
Requires-Dist: nardial[dialogflow,elevenlabs,google-tts,openai,webserver]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Dynamic: license-file

# NarDialPy

**NarDialPy** is a Python package for building and running **narrative-driven, structured dialog systems** — designed for social robots and conversational agents.

It lets you author complete conversations declaratively in JSON, then drive them from Python using voice, NLU, LLM, and browser-based screen services. The package handles session flow, branching logic, topic tracking, personalization, and screen output so you can focus on what the robot says and how conversations unfold.

---

## Table of Contents

1. [What is nardial?](#what-is-nardial)
2. [Prerequisites & Setup](#prerequisites--setup)
3. [Providers & Initialization](#providers--initialization)
4. [Defining Dialogs in JSON](#defining-dialogs-in-json)
   - [Dialog Structure](#dialog-structure)
   - [Dialog Types](#dialog-types)
     - [Custom dialog types](#custom-dialog-types)
   - [Move Types](#move-types)
   - [Key JSON Attributes](#key-json-attributes)
5. [Demos / Creating a Session](#demos--creating-a-session)
6. [Development](#development)

---

## What is nardial?

The `nardial` package provides the building blocks for authoring and executing multi-turn conversations:

| Component | Description |
|---|---|
| **Dialog JSON** | Conversations are written as structured JSON files. Each file holds one or more *dialogs*, each containing a sequence of *moves* that the robot performs. |
| **Session Manager** | Loads your dialog JSON, resolves a session agenda, and runs dialogs in order — checking dependencies and tracking state. |
| **ConversationAgent** | The runtime bridge to the hardware: it calls TTS, STT, LLM, and motion services on your chosen device. |
| **Screen Provider** | Optional browser-based display layer for transcripts, images, videos, HTML, buttons, and web input. |
| **Dialog Logic** | Checks eligibility rules (dependencies, variable requirements) before executing each dialog. |

Typical use case:

- A social robot (Pepper, NAO, desktop agent) runs a structured conversation with a child or adult participant.
- The conversation is broken into named dialog blocks (greeting, story, chitchat, goodbye) authored in JSON.
- Python code wires up the device, credentials, and agenda — the JSON drives the actual content and branching.

Additional community demos are in the [SIC Applications repository](https://github.com/Social-AI-VU/sic_applications/tree/main/demos/nardial).

---

## Prerequisites & Setup

### 1. Python IDE

Recommended: [PyCharm](https://www.jetbrains.com/help/pycharm/installation-guide.html) or [VS Code](https://code.visualstudio.com/download)

### 2. Python

- Version: **3.12**
- Download: https://www.python.org/downloads/release/python-3120/
- ⚠️ Ensure Python is added to your system `PATH`

### 3. In your project, create and activate your python virtual environment 
```bash
# Windows
cd your_project_folder
python -m venv venv_myproject
venv_myproject\Scripts\activate

# macOS / Linux
cd your_project_folder
python -m venv venv_myproject
source venv_myproject/bin/activate
```

### 4. Install NarDial

Install the core package, then add extras for each service you intend to use:

```bash
pip install nardial
```

| Extra | Enables | Install command |
|---|---|---|
| `google-tts` | Google Cloud Text-to-Speech | `pip install "nardial[google-tts]"` |
| `elevenlabs` | ElevenLabs Text-to-Speech | `pip install "nardial[elevenlabs]"` |
| `dialogflow` | Google Dialogflow NLU | `pip install "nardial[dialogflow]"` |
| `openai` | OpenAI GPT | `pip install "nardial[openai]"` |
| `all` | All of the above | `pip install "nardial[all]"` |

For robot devices, install the matching SIC device extra directly:

```bash
pip install "social-interaction-cloud[alphamini]"   # Alphamini
```

Pepper and NAO are included in the base SIC package.


### 5. Configure Credentials
 * Create a `conf` folder at the root of your project.
 * Create a Google cloud project (for Dialogflow and google-tts services) and save the generated keyfile to `conf/google/google_keyfile.json`. Instructions [here](https://socialrobotics.atlassian.net/wiki/spaces/CBSR/pages/2205155343/Getting+a+google+dialogflow+key).
 * Create an OpenAI account, and generate an [API key](https://developers.openai.com/api/docs/quickstart) and save it to `conf/openai/.openai_env` with the following content:
```bash
OPENAI_API_KEY="your key"
```

> ⚠️ Never commit credential files to version control.

### 6. Start Required Services

Run these in separate terminals before starting any demo:

```bash
# Windows
conf/redis/redis-server.exe conf/redis/redis.conf

# macOS / Linux
redis-server conf/redis/redis.conf
```

```bash
run-dialogflow
run-google-tts
run-gpt
```

---

## Providers & Initialization

NarDialPy is built around a set of provider protocols. Each protocol defines a role (device, TTS, NLU, LLM, vector store) and multiple concrete implementations are available. You pick one implementation per role, instantiate it, and pass everything into `ConversationAgent` or `SessionManager`.

### Available Providers

| Role | Provider | Import path                                  | Requires |
|---|---|----------------------------------------------|---|
| **Device** | `DesktopAdapter` | `nardial.providers.device.desktop`           | base |
| | `PepperAdapter` | `nardial.providers.device.pepper`            | base |
| | `NaoAdapter` | `nardial.providers.device.nao`               | base |
| | `AlphaminiAdapter` | `nardial.providers.device.alphamini`         | `social-interaction-cloud[alphamini]` |
| | `ReachyMiniAdapter` | `nardial.providers.reachy_mini`              | `social-interaction-cloud[reachy-mini]` | 
| **TTS** | `GoogleTTSProvider` | `nardial.providers.tts.google`               | `nardial[google-tts]` |
| | `ElevenLabsTTSProvider` | `nardial.providers.tts.elevenlabs`           | `nardial[elevenlabs]` |
| | `NaoqiTTSProvider` | `nardial.providers.tts.naoqi`                | base (uses device's built-in TTS) |
| | `NullTTSProvider` | `nardial.providers.tts.null`                 | base (prints to terminal) |
| **NLU** | `DialogflowNLUProvider` | `nardial.providers.nlu.dialogflow`           | `nardial[dialogflow]` |
| | `WrittenKeywordNLUProvider` | `nardial.providers.nlu.written_keyword`      | base (keyboard input) |
| **LLM** | `OpenAIGPTProvider` | `nardial.providers.llm.openai_gpt`           | `nardial[openai]` |
| | `EchoLLMProvider` | `nardial.providers.llm.echo`                 | base (echoes user input) |
| **Screen** | `ScreenProvider` / `SICScreenAdapter` / `PepperTabletScreenAdapter` | `nardial.providers.screen`                   | browser display via SIC webserver |
| **Vector store** | `RedisVectorStoreProvider` | `nardial.providers.vector_store.redis_store` | base + running Redis |
| | `NullVectorStoreProvider` | `nardial.providers.vector_store.null`        | base |

---

### Minimal setup (no external services)

Good for local development and testing — all I/O goes through the terminal:

```python
import logging
from sic_framework.devices.desktop import Desktop

from nardial.providers.device.desktop import DesktopAdapter
from nardial.providers.tts.null import NullTTSProvider
from nardial.providers.nlu.written_keyword import WrittenKeywordNLUProvider
from nardial.conversation_agent import ConversationAgent

desktop = Desktop()
device = DesktopAdapter(desktop)
device.setup(logger=logging.getLogger())

agent = ConversationAgent(
    device=device,
    tts_provider=NullTTSProvider(),
    nlu_provider=WrittenKeywordNLUProvider(),
)
```

---

### Desktop with cloud services

```python
import json, logging
from sic_framework.devices.desktop import Desktop
from sic_framework.services.dialogflow.dialogflow import DialogflowConf

from nardial.providers.device.desktop import DesktopAdapter
from nardial.providers.tts.google import GoogleTTSProvider, GoogleTTSConf
from nardial.providers.tts.cacher import TTSCacher
from nardial.providers.nlu.dialogflow import DialogflowNLUProvider
from nardial.providers.llm.openai_gpt import OpenAIGPTProvider
from nardial.conversation_agent import ConversationAgent

desktop = Desktop()
device = DesktopAdapter(desktop)
device.setup(logger=logging.getLogger())

tts = GoogleTTSProvider(
    conf=GoogleTTSConf(speaking_rate=0.9, google_tts_voice_name="en-US-Neural2-F"),
    device=device,
    keyfile_path="conf/google/google_keyfile.json",
    tts_cacher=TTSCacher(tts_cache_dir="tts_cache"),
)

nlu = DialogflowNLUProvider(
    conf=DialogflowConf(keyfile_json=json.load(open("conf/google/google_keyfile.json"))),
    mic=desktop.mic,
)

llm = OpenAIGPTProvider(api_key="<YOUR_OPENAI_KEY>")

agent = ConversationAgent(
    device=device,
    tts_provider=tts,
    nlu_provider=nlu,
    llm_provider=llm,
)
```

---

### Pepper robot

Swap the device adapter and TTS provider — everything else stays the same:

```python
import logging
from sic_framework.devices import Pepper

from nardial.providers.device.pepper import PepperAdapter
from nardial.providers.tts.naoqi import NaoqiTTSProvider

pepper = Pepper(ip="<PEPPER_IP>")
device = PepperAdapter(pepper)
device.setup(logger=logging.getLogger())

tts = NaoqiTTSProvider(device=device, language="en")
```

Then pass `device` and `tts` to `ConversationAgent` as above.

---

### Pause and skip controls

Bind keys to pause the interaction or to skip a listen that takes too long by passing `controls` in an `InteractionConfig`. Keys work globally (the terminal does not need focus), with or without `SessionManager`:

```python
from nardial.interaction_orchestrator import InteractionConfig

agent = ConversationAgent(
    ...,  # device and providers as in the setups above
    int_config=InteractionConfig(
        controls={
            "start_paused": True,          # wait for the pause key before starting
            "toggle_pause": ["space"],     # pause / resume
            "skip_listen": ["s"],          # stop a listen that takes too long
        },
    ),
)
```

| Setting | Effect |
|---|---|
| `toggle_pause` | Pauses the interaction. Speech in progress is stopped and spoken again from the beginning after resume. Listening in progress is stopped, what was heard so far is kept, and the dialog continues after the listen once resumed. Works in every dialog type, including LLM dialogs. |
| `skip_listen` | Stops the current listen early. What was said so far is still recognised and used as the answer: Dialogflow gets a `StopListeningMessage`, and local Whisper gets an end-of-audio signal and then transcribes what it recorded (this can take a few seconds; Whisper waits up to 30 s for it, which a faster model can lower with `LocalWhisperNLUProvider(..., stop_grace_seconds=5)`). The written keyword provider cannot stop early and gives up after 3 seconds with an empty answer. |
| `start_paused` | `true` makes the interaction wait for `toggle_pause` before it starts. |

A question whose listen was stopped by a pause or skip is not asked again, even when the answer is empty: the dialog continues with whatever was heard, and an empty answer follows the move's `default_outcome`.

Key names are single characters (`"p"`) or [pynput key names](https://pynput.readthedocs.io/en/latest/keyboard.html#pynput.keyboard.Key) (`"space"`, `"enter"`, `"f5"`). Moves that are not speech or listening (e.g. `timed_wait`, `wait_for_button`) are not paused; the pause takes effect at the next speech or listen.

---

### Using SessionManager

`SessionManager` wraps `ConversationAgent` and adds dialog loading, eligibility checking, and session state:

```python
from nardial.session_manager import SessionManager

manager = SessionManager(
    agent=agent,
    dialog_file="dialogs/my_dialogs.json",
    participant_id="user_42",
)
manager.run()
```

If a previous run crashed mid-session, pass `resume=True` to continue that same incomplete session (one whose `ended_at` is still `None`) instead of starting a new one — dialogs already completed in it are skipped:

```python
manager = SessionManager(
    agent=agent,
    dialog_file="dialogs/my_dialogs.json",
    participant_id="user_42",
    resume=True,
)
manager.run()
```

See [Developer Flow](docs/DEVELOPER_README.md) for more on session/state handling.

---

## Defining Dialogs in JSON

All conversation content lives in a JSON file (or directory of JSON files). The file is a JSON array of dialog objects.

### Dialog Structure

Every dialog has the following shared fields:

| Field | Type | Required | Description |
|---|---|---|---|
| `id` | string | ✅ | Unique identifier referenced in `session_agenda` and `dependencies` |
| `type` | string | ✅ | Dialog type: `"functional"`, `"chitchat"`, `"narrative"`, `"llm_based"`, or a [custom type](#custom-dialog-types) |
| `moves` | array | ✅ | Ordered list of move objects the robot will perform |
| `dependencies` | array of strings | | Dialog IDs that must have been completed before this dialog may run |
| `variable_dependencies` | array | | Variables that must exist in the user model before this dialog may run |
| `prerequisites` | array | | Actions run just before the dialog's first move: call a registered Python function, or set user-model variables (see [below](#optional-top-level-prerequisites)) |

Example: 
```json
{
  "id": "greeting",
  "type": "functional",
  "functional_type": "greeting",
  "moves": [
    { "type": "say", "text": "Hi! I am your robot assistant." }
  ]
}
```

---

### Dialog Types

#### Optional Top-Level `characters`

Dialogs can define reusable character voices:

```json
{
  "characters": {
    "narrator": {
      "voice_settings": {
        "voice_id": "KTPVrSVAEUSJRClDzBw7",
        "language": "en"
      }
    }
  }
}
```

- `characters` is optional.
- Each key is a character name.
- Each value must contain a `voice_settings` object.
- `voice_settings` is validated at runtime against the active TTS provider.

#### Optional Top-Level `prerequisites`

Dialogs can list actions to run just before their first move, each time the dialog runs:

```json
{
  "id": "multiplication",
  "type": "chitchat",
  "prerequisites": [
    { "execute": "choose_math_level", "args": [true, false] , "skip_dialog":  true},
    { "set_variable": { "math_left": 4, "math_right": 3 } }
  ],
  "moves": [
    { "type": "say", "text": "What is %math_left% times %math_right%?" }
  ]
}
```

Each entry does one of two things:

| Key | Value | What happens |
|---|---|---|
| `execute` | name of a registered function | Calls that Python function. Optional `args`: a list is passed as separate arguments (`[3, 4]` calls `f(3, 4)`), any other value as a single argument, and without `args` the function is called with none. |
| `set_variable` | object | Merges the object into the user model, so its values can be used in `%variable%` placeholders and later dialogs. |

In every prerequisite, `"skip_dialog` can be set to `true` of `false`, based on whether the dialog needs to be skipped if the prerequisite cannot be completed. `"skip_dialog:true` means that the dialog will be skipped if the prerequisite is not completed. If `"skip_dialog: false` is chosen, the dialog will still run, even if the prerequisite could not be completed. The default is `false`.
Functions are registered in Python under the name used in `execute`, before the session runs:

```python
from nardial.mini_dialogs import MiniDialog

@MiniDialog.register("choose_math_level")
def choose_math_level(last_correct, needed_help):
    if last_correct and not needed_help:
        level = 2
    else:
        level = 1
    return level

```

- Prerequisites run in the order listed, after the dialog has been picked. They can't change whether the dialog itself is eligible, only what happens once it runs.
- Registered functions must be regular (not `async`) functions. They get only the `args` from the JSON, not the dialog or the user model.
- If a function isn't registered or raises an error, the error is printed and the remaining prerequisites and the dialog still run.

---

#### 1.`functional`

Utility dialogs for session management — greetings, farewells, and structural transitions.

| Extra field | Type | Required | Description |
|---|---|---|---|
| `functional_type` | string | ✅ | `"greeting"` or `"farewell"` |

```json
{
  "id": "welcome",
  "type": "functional",
  "functional_type": "greeting",
  "moves": [
    { "type": "say", "text": "Hello! What is your name?" },
    {
      "type": "ask_open",
      "text": "Please tell me your name.",
      "set_variable": "first_name"
    }
  ]
}
```

---

#### 2. `chitchat`

Short, theme-based conversations on everyday topics. Chitchat dialogs can be seeded with topics of interest so the system selects contextually relevant ones.

| Extra field | Type | Required | Description |
|---|---|---|---|
| `theme` | string | ✅ | Broad category (e.g. `"nature"`, `"animals"`, `"robots"`) |
| `topics` | array of strings | | Specific keywords used to drive relevance matching |

```json
{
  "id": "favorite_animal",
  "type": "chitchat",
  "theme": "animals",
  "topics": ["animals", "pets"],
  "moves": [
    {
      "type": "ask_open",
      "text": "What is your favorite animal?",
      "set_variable": "favorite_animal",
      "add_interest_from_answer": true
    },
    { "type": "say", "text": "I like %favorite_animal% too!" }
  ]
}
```

---

#### 3. `narrative`

Story-based dialogs that belong to a named thread and must be delivered in a specific order. Use `position` to sequence them and `dependencies` to enforce ordering.

| Extra field | Type | Required | Description |
|---|---|---|---|
| `thread` | string | ✅ | Story thread name (e.g. `"dreams"`) — groups related narrative dialogs |
| `position` | integer | ✅ | Order within the thread (1, 2, 3, …) |

```json
{
  "id": "dream_intro",
  "type": "narrative",
  "thread": "dreams",
  "position": 1,
  "moves": [
    { "type": "say", "text": "Sometimes I make up stories when I'm turned off." },
    {
      "type": "ask_yesno",
      "text": "Do you ever dream?",
      "set_variable": "dreams_yesno",
      "outcomes": { "yes": "dreams_yes" },
      "default_outcome": "dreams_no"
    },
    {
      "type": "branch",
      "on": "outcome",
      "cases": {
        "dreams_yes": [
          { "type": "say", "text": "That's great! Dreams are fascinating." }
        ],
        "dreams_no": [
          { "type": "say", "text": "No problem, I'll tell you about mine!" }
        ]
      }
    }
  ]
}
```

---

#### 4. `llm_based`

A fully LLM-driven dialog where the robot and user engage in a free-form multi-turn exchange guided by a system prompt. No `moves` array is needed — the LLM generates all responses.

| Extra field | Type | Required | Description |
|---|---|---|---|
| `prompt` | string | ✅ | System prompt guiding the LLM's behavior |
| `max_turns` | integer | | Maximum back-and-forth turns (default: 5) |
| `speak_first` | boolean | | If `true` (default), the robot speaks first; if `false`, it listens first |
| `duration` | number | | Time limit in seconds for the whole exchange |
| `quit_phrases` | array of strings | | User utterances that end the exchange early |
| `quit_signal` | string | | Token the LLM can embed to signal it wants to end (default: `"<<QUIT>>"`) |
| `rag_enabled` | boolean | | Enable Retrieval-Augmented Generation (RAG). When `true`, the LLM retrieves context from a vector index before generating responses. Requires configuring a compatible RAG backend through `social-interaction-cloud`. |
| `index_name` | string | Required when `rag_enabled` is `true` | Name of the RAG index to query |

```json
{
  "id": "free_chat",
  "type": "llm_based",
  "prompt": "You are a friendly robot. Chat warmly with the child about their day. Ask follow-up questions. End with <<QUIT>> when the topic is exhausted.",
  "max_turns": 6,
  "speak_first": true,
  "quit_signal": "<<QUIT>>",
  "quit_phrases": ["goodbye", "stop", "done"],
  "moves": []
}
```

#### Custom dialog types

You can define your own dialog type in JSON. A custom type is based on an existing type and changes the rules that decide when its dialogs may run.

Type definitions go in their own JSON file (or a directory of them), separate from your dialogs. Mixing them is an error: a type definition in a dialog file, or a dialog in a types file, is reported and skipped. Pass the types file to `SessionManager` as `dialog_types_path`. It is loaded before the dialogs, so dialogs can use the new types:

```python
manager = SessionManager(
    session_agenda=[...],
    agent=agent,
    dialog_types_path="dialog_types/my_types.json",
    dialog_json_path="dialog_json/my_dialogs.json",
)
```

Without `SessionManager`, call `load_dialog_types(path)` (from `nardial.authoring`) before loading the dialogs. Within the types file(s), the order of definitions doesn't matter, even when one type extends another.

A type definition looks like this:

```json
{
  "define_type": "repeatable_chitchat",
  "extends": "chitchat",
  "description": "Chitchat that may come back in later sessions, but runs at most once per session.",
  "remove_rules": ["exclude_if_seen"],
  "add_rules": [{ "rule": "exclude_if_seen", "scope": "session" }]
}
```

| Field | Type | Required | Description |
|---|---|---|---|
| `define_type` | string | ✅ | Name of the new type, used as `type` in dialogs |
| `extends` | string | ✅ | Type it is based on: a built-in type or another custom type |
| `description` | string | | Free-text description |
| `remove_rules` | array of strings | | Names of the base type's rules to drop |
| `add_rules` | array | | Rules to add: a rule name, or an object with `"rule"` plus that rule's settings |

Dialogs of the new type use the same fields as dialogs of the type it extends:

```json
{
  "id": "music_lately",
  "type": "repeatable_chitchat",
  "topics": ["music"],
  "moves": [{ "type": "say", "text": "What music have you been listening to lately?" }]
}
```

They are also picked by the same agenda slots. For example, a `chitchat_slot` can choose a `repeatable_chitchat` dialog.

**Built-in types and their rules:**

| Type | Rules |
|---|---|
| `functional` | `dependency_met` |
| `narrative` | `exclude_if_seen`, `dependency_met`, `variable_dependency_met`, `narrative_ordering` |
| `chitchat` | `exclude_if_seen`, `dependency_met`, `variable_dependency_met` |
| `llm_based` | `exclude_if_seen`, `dependency_met`, `variable_dependency_met` |

**Available rules.** A dialog may run only when all of its type's rules allow it.

| Rule | Settings | Allows the dialog to run when |
|---|---|---|
| `exclude_if_seen` | `scope`: `"participant"` (default) or `"session"` | it hasn't been completed before (by this participant, or in this session) |
| `dependency_met` | | every dialog in its `dependencies` has been completed |
| `variable_dependency_met` | | every required variable in its `variable_dependencies` is set in the user model |
| `narrative_ordering` | | every earlier `position` in its `thread` has been completed |

**Your own rules.** Any other rule is written in Python and registered under a name, which `add_rules` can then use. Settings from the JSON are passed to the constructor:

```python
from nardial.eligibility import EligibilityRule, register_rule

@register_rule("from_session")
class FromSessionRule(EligibilityRule):
    def __init__(self, min):
        self.min = min

    def is_eligible(self, dialog, context):
        # context.session_index: session number starting at 1, or None when unknown
        return context.session_index is not None and context.session_index >= self.min
```

```json
{ "define_type": "returning_chitchat", "extends": "chitchat", "add_rules": [{ "rule": "from_session", "min": 2 }] }
```

Import the module that registers the rule before loading your dialogs. A rule can read `context.user_model`, `context.completed_ids`, `context.session_completed_ids`, `context.session_index` and `context.registry`. See `docs/DEVELOPER_README.md` for more, including defining types directly in Python.

---

### Move Types

Moves are the individual steps inside a dialog's `moves` array. The robot executes them in order.

---

#### `say`

Speaks a piece of text. Variable placeholders (`%variable_name%`) are replaced at runtime.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"say"` |
| `text` | string | ✅ | Text to speak. Use `%var%` to insert stored variables. |
| `character` | string | | Optional character name from top-level `characters`; if omitted, default provider voice settings are used. |

```json
{ "type": "say", "text": "Nice to meet you, %first_name%!" }
```

---

#### `say_options`

Speaks one randomly chosen response from a predefined list.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"say_options"` |
| `options` | array of strings | ✅ | Candidate utterances to choose from at runtime |
| `character` | string | | Optional character name from top-level `characters` |

```json
{ "type": "say_options", "options": ["Hi!", "Hello!", "Hey there!"] }
```

---

#### `ask_open`

Asks a free-text question and listens for any spoken reply. The answer can be stored in a variable and used to drive branching.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"ask_open"` |
| `text` | string | ✅ | The question to ask |
| `set_variable` | string | | Stores the extracted answer in the user model under this name |
| `outcomes` | object | | Maps answer values to outcome labels. Use `"*"` as a wildcard for any non-empty answer. |
| `default_outcome` | string | | Outcome label when no answer or no match is found |
| `add_interest_from_answer` | boolean | | If `true`, adds the answer to the user's topics of interest |
| `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence after the user replies |
| `llm_extract_entity` | boolean | | If `true`, the LLM extracts the key entity from the answer (e.g. `"dog"` from `"my favorite animal is a dog"`) before it is stored; unrelated answers store nothing. Default `false` |
| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |

```json
{
  "type": "ask_open",
  "text": "What is your favorite color?",
  "set_variable": "favorite_color",
  "add_interest_from_answer": true
}
```

---

#### `ask_yesno`

Asks a yes/no question. The detected intent (`"yes"`, `"no"`, `"dontknow"`) drives branching.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"ask_yesno"` |
| `text` | string | ✅ | The yes/no question to ask |
| `set_variable` | string | | Stores the answer in the user model |
| `outcomes` | object | | Maps `"yes"` / `"no"` / `"dontknow"` to outcome labels |
| `default_outcome` | string | | Outcome label used as fallback |
| `add_interest` | string | | Topic added to interest list when the user answers `"yes"` |
| `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |

```json
{
  "type": "ask_yesno",
  "text": "Do you like dogs?",
  "set_variable": "likes_dogs",
  "add_interest": "dogs",
  "outcomes": { "yes": "likes_dogs_yes" },
  "default_outcome": "likes_dogs_no"
}
```

---

#### `ask_options`

Presents a multiple-choice question. The selected option drives branching and can be stored as a variable.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"ask_options"` |
| `text` | string | ✅ | The question to ask |
| `options` | array of strings | ✅ | The choices presented to the user |
| `set_variable` | string | | Stores the selected option in the user model |
| `outcomes` | object | | Maps option values to outcome labels |
| `default_outcome` | string | | Outcome label when nothing matches |
| `add_interest_from_variable` | string | | After storing, adds the named variable's value as a topic of interest |
| `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |

```json
{
  "type": "ask_options",
  "text": "Which activity do you prefer — reading, walking, or cooking?",
  "options": ["reading", "walking", "cooking"],
  "set_variable": "preferred_activity",
  "outcomes": {
    "reading": "chose_reading",
    "walking": "chose_walking",
    "cooking": "chose_cooking"
  },
  "default_outcome": "chose_other"
}
```

---

#### `ask_llm`

Starts a multi-turn LLM-driven exchange *within* an otherwise scripted dialog. Useful for a single free-form segment inside a structured conversation.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"ask_llm"` |
| `prompt` | string | ✅ | System prompt for the LLM |
| `max_turns` | integer | | Maximum turns (default: 5) |
| `set_variable` | string | | Stores the user's last reply in the user model |
| `quit_phrases` | array of strings | | User phrases that end the exchange early |
| `quit_signal` | string | | Token the LLM emits to signal it wants to stop |

```json
{
  "type": "ask_llm",
  "prompt": "Ask one follow-up question to help the user commit to their plan for today. Keep it short and friendly.",
  "max_turns": 1,
  "set_variable": "plan_commitment"
}
```

---

#### `branch`

Selects and executes a list of sub-moves based on the current outcome or the value of a user model variable.

| Field       | Type | Required | Description                                                                                                                                                                                        |
|-------------|---|---|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `type`      | string | ✅ | `"branch"`                                                                                                                                                                                         |
| `on`        | string | ✅ | `"outcome"` to branch on the last question's result, `"variables"` to branch based on whether to variabels in the user model have the same value, or a variable name to branch on its stored value |
| `cases`     | object | ✅ | Maps condition values to arrays of sub-moves                                                                                                                                                       |
| `variables` | list | | In case of branching on `"variables"`, a list of the variables that need to be compared | 

```json
{
  "type": "branch",
  "on": "outcome",
  "cases": {
    "chose_reading": [
      { "type": "say", "text": "Reading is a wonderful way to relax." }
    ],
    "chose_walking": [
      { "type": "say", "text": "A walk sounds refreshing!" }
    ]
  }
}
```

Branching on a stored variable (e.g. to react to an answer from an earlier dialog):

```json
{
  "type": "branch",
  "on": "energy_level",
  "cases": {
    "high": [{ "type": "say", "text": "Start with a longer session." }],
    "low":  [{ "type": "say", "text": "Begin with just 10 calm minutes." }]
  }
}
```

Branching on comparison of two stored variables (e.g. to check whether a given answer is correct). Chooses branch "true" if the values of the variables are the same, branch "false" if not.

```json
{
  "type": "branch",
  "on": "variables",
  "variables": ["given_answer", "correct_answer"],
  "cases": {
    "true": [{ "type":  "say", "text":  "That's correct!"}],
    "false": [{ "type":  "say", "text":  "That's incorrect. Let's try again."}]
  }
}
```

---

#### `play`

Plays an audio file through the device's speakers.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"play"` |
| `audio` | string | ✅ | Path to the audio file (`.wav` or `.mp3`) |

```json
{ "type": "play", "audio": "audio/chime.wav" }
```

---

#### `motion_sequence`

Plays a predefined motion sequence on the robot (Pepper / NAO).

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"motion_sequence"` |
| `motion_sequence` | string | ✅ | Path or name of the motion sequence file |

```json
{ "type": "motion_sequence", "motion_sequence": "motions/Stand/Emotions/Positive/Happy_1" }
```

---

#### `animation`

Triggers a named animation behavior on the robot.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"animation"` |
| `animation_name` | string | ✅ | Name of the animation to play |

```json
{ "type": "animation", "animation_name": "animations/Stand/Gestures/Enthusiastic_4" }
```

---

#### `timed_wait`

Pause execution for a fixed duration before proceeding to the next move. Useful to add natural pauses or give the user time to look at the screen.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"timed_wait"` |
| `duration_seconds` | number | ✅ | Seconds to wait. (Alias: `duration` is also accepted.) |

```json
{ "type": "timed_wait", "duration_seconds": 2.5 }
```

---

#### `wait_for_web_input`

Suspend dialog execution until a matching web input event arrives or until a timeout elapses. The web UI can present buttons or a short input and emit `web_input` events which this move listens for.

| Field | Type | Required | Description                                                                     |
|---|---|---|---------------------------------------------------------------------------------|
| `type` | string | ✅ | `"wait_for_web_input"`                                                          |
| `prompt` | string | | Hint text shown in the web UI (not spoken)                                      |
| `options` | array of strings | | Accepted `value` strings from the web event (if omitted, any value is accepted) |
| `timeout` | number | | Seconds to wait before falling back (omit for indefinite wait)                  |
| `outcomes` | object | | Maps option value → outcome label for branching                                 |
| `default_outcome` | string | | Outcome used on timeout or when no event matches                                |
| `set_variable` | string | | Stores the user's choice in the user model                                      |

```json
{
  "type": "wait_for_web_input",
  "prompt": "Choose a sticker to show",
  "options": ["smile","thumbs","surprised"],
  "timeout": 15,
  "outcomes": {"smile": "picked_smile"},
  "default_outcome": "no_choice",
  "set_variable": "sticker"
}
```

---

#### `show_image`

Display an image on the connected screen (browser / tablet). Accepts a local path relative to the static assets directory or a full URL.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"show_image"` |
| `src` | string | ✅ | Path or URL to the image (aliases: `image`, `path`) |

```json
{ "type": "show_image", "src": "assets/images/robot_pet.png" }
```

---

#### `show_video`

Display a video on the screen. Can accept a local file path or an embeddable URL (e.g. a YouTube embed link).

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"show_video"` |
| `src` | string | ✅ | Video file path or embeddable URL (aliases: `video`, `path`) |

```json
{ "type": "show_video", "src": "https://www.youtube.com/embed/VIDEO_ID" }
```

---

#### `show_iframe`

Embed an external web page inside an iframe on the screen. Use this for interactive web content served from a trusted origin.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"show_iframe"` |
| `url` | string | ✅ | URL to embed (aliases: `src`, `iframe_url`) |

```json
{ "type": "show_iframe", "url": "https://example.com/mini-game" }
```

---

#### `show_html`

Render a raw HTML snippet directly into the display area. The frontend inserts this via innerHTML — treat content as trusted and avoid injecting untrusted user data.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"show_html"` |
| `html` | string | ✅ | HTML snippet to render |

```json
{ "type": "show_html", "html": "<div class=\"card\">Hi there!</div>" }
```

---

#### `black_screen`

Clear the display (show a blank/black screen). No parameters — the next display move will restore content.

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | ✅ | `"black_screen"` |

```json
{ "type": "black_screen" }
```

---

#### `keyboard_input`

Display keyboard input on screen. Suspend dialog execution until keyboard input is given or until a timeout elapses.

| Field | Type | Required | Description                                                    |
|---|---|---|----------------------------------------------------------------|
| `type` | string | ✅ | `"keyboard_input"`                                             |
| `prompt` | string | | Hint text shown in the web UI (not spoken)                     |
| `timeout` | number | | Seconds to wait before falling back (omit for indefinite wait) |
| `outcomes` | object | | Maps option value → outcome label for branching                |
| `default_outcome` | string | | Outcome used on timeout or when no event matches               |
| `set_variable` | string | | Stores the user's choice in the user model                     |

```json
{
  "type": "keyboard_input",
  "prompt": "How old are you?",
  "timeout": 15,
  "default_outcome": "no_age_given",
  "set_variable": "age"
}
```

---

#### `go_to_dialog`

Move to a specified new minidialog.

| Field             | Type | Required | Description                                                    |
|-------------------|---|----------|----------------------------------------------------------------|
| `type`            | string | ✅        | `"go_to_dialog"`                                               |
| `dialog_id`       | string |✅          | ID of the dialog to go to                                      |

```json
{
  "type": "go_to_dialog",
  "dialog_id": "favorite_animal"
}
```

---

### Key JSON Attributes

| Attribute | Where used | Description |
|---|---|---|

---

## Demos / Creating a Session

All you need is a minimal Python script that wires up the device, loads the dialog JSON, and runs the session. You can follow the included demos to get started quickly.

Ready-to-run demos are included in the `examples/` directory:
* Demo 1 — General Conversation (`demo_general_conversation.py`): A simple four-step conversation using a mix of narrative and functional dialogs
* Demo 2 — Structured Conversation (`demo_structured_conversation.py`): A more complete example that demonstrates all dialog types and move types, including `ask_llm`, `play`, `motion_sequence`, and `animation`
* Demo 3 — Screen Display (`demo_screen_provider.py`): Shows the browser-based screen UI with transcripts, images, iframes, HTML snippets, buttons, and text input
* Demo 4 — Pepper Tablet (`demo_pepper_tablet.py`): Uses the same screen UI on Pepper's tablet through the SIC webserver
* Demo 5 — Agenda System (`demo_agenda_system.py`): Uses every agenda-item type (`narrative_slot`, `chitchat_slot`, `functional_slot`, `llm_dialog_ref`, plain dialog ids) so `SessionManager` picks the next dialog dynamically instead of following a fixed list — see the [Agenda System Guide](docs/AGENDA_SYSTEM_GUIDE.md) for a plain-language walkthrough
* Demo 6 — Custom Dialog Type (`demo_custom_dialog_type.py`): Writes a custom eligibility rule in Python and defines two new dialog types in JSON (see [Custom dialog types](#custom-dialog-types)): a chitchat that comes back every session, and one that only runs for participants who said they have a pet

You can find additional demos in the [SIC Applications repository](https://github.com/Social-AI-VU/sic_applications/tree/main/demos/nardial)

---

## Development

For an architecture-level explanation of how JSON dialogs become running conversations, see [Developer Flow](docs/DEVELOPER_README.md). If you're building an application and want to understand dynamic session agendas rather than runtime internals, see the [Agenda System Guide](docs/AGENDA_SYSTEM_GUIDE.md) instead.

Run tests from the repository root:

```bash
python -m pytest -q
```
