🧬 AlphaAvatar Persona Plugin

The Persona Plugin provides user identity recognition and profile management for AlphaAvatar.

🧩 Module Overview

The plugin is responsible for:

• identifying users through face and speaker embeddings;
• associating temporary face, speaker, and session identities with stable users;
• maintaining user profile attributes;
• exposing the active user persona to the Avatar Runtime;
• publishing identity-related annotations into PerceptionRuntime.

Persona processing is divided into independent paths:

Visual Observation
    └── → Face Recognition

Speech Observation
    └── → Speaker Recognition

Conversation Context
    └── → Persona Profile

The resulting identity and profile information can be used by Memory, prompting, retrieval, and other runtime modules.

🔍 Identity Processing

👁️ Face Recognition

The face stream consumes visual observations from PerceptionRuntime.

Video / Screen Observation
            ↓
      PerceptionRuntime
            ↓
       FaceStreamWrapper
            ↓
     FaceAnalysisRunner
            ↓
 Face Embedding and Attributes
            ↓
      User Identity Match

Face processing:

• samples frames independently from the shared visual stream;
• performs face detection and embedding extraction;
• matches embeddings against known user profiles;
• updates face attributes and identity associations;
• publishes face annotations back to the original observation;
• optionally renders face bounding boxes into the annotated payload view.

Raw video observations remain unchanged. Rendered overlays are stored in the separate annotated payload view.

🎤 Speaker Recognition

The speaker stream consumes routed speech observations rather than subscribing directly to LiveKit audio.

PerceptionRuntime.audio
          ↓
AudioActivityProcessor
          ↓
PerceptionRuntime.speech
          ↓
     Speaker Stream
          ↓
 Speaker Embedding and Match

Speaker processing uses rolling speech windows to:

• create speaker embeddings;
• match speech against known users;
• update voice identity attributes;
• associate temporary speaker identities with stable user profiles.

Raw audio frames are used only during runtime processing and are not stored as persona profiles.

🧩 Supported Components

Persona profiles may contain:

• stable user identifiers;
• face and speaker embeddings;
• face and voice attributes;
• profile text;
• temporary identity aliases;
• session and runtime metadata.

🔗 Module Relationships

PerceptionRuntime
    ├── → Face observations
    └── → Speech observations

Router Plugin
    └── → Routed speech stream

Persona Plugin
    ├── → Active user identity
    ├── → Persona prompt context
    ├── → Face and speaker annotations
    └── → Identity aliases

Memory Plugin
    └── → User ownership and identity-aware retrieval

📦 Installation

pip install alpha-avatar-plugins-persona

The plugin is loaded through the AlphaAvatar configuration.