spaceverseai 1.0.0

Advanced Real-Time Voice Chat System with Emotion Detection and AI Integration

SpaceVerse AI - Advanced Real-Time Voice Chat System

🌟 Overview

SpaceVerse AI is a cutting-edge real-time voice chat system that combines OpenAI's GPT-4o with advanced emotion detection, user preference analysis, and dynamic personalization. The system provides an intelligent, emotionally-aware conversational AI that adapts to users' communication styles and emotional states in real-time.

🚀 Key Features

• Real-Time Voice Communication: Seamless voice chat with OpenAI's GPT-4o
• Facial Emotion Detection: Real-time emotion analysis using deep learning models
• User Preference Learning: Dynamic adaptation to individual communication styles
• Threat Detection: Advanced safety monitoring for crisis intervention
• Multi-Language Support: Support for 12+ languages
• Session Persistence: Intelligent session management with metadata tracking
• WebSocket Architecture: Real-time bidirectional communication
• Emotional Intelligence: AI responses that match detected emotions
• Onboarding System: Dynamic question generation for personalization

🛠️ Installation

Quick Installation (Recommended)

Install SpaceVerse AI directly from PyPI:

pip install spaceverseai

That's it! All dependencies will be installed automatically.

🚀 Quick Start

Async Usage

import asyncio
from spaceverseai import SpaceverseAIClient, VoiceType, LanguageCode

async def main():
    async with SpaceverseAIClient() as client:
        # Check server health
        health = await client.health_check()
        print(f"Server status: {health.status}")
        
        # Connect to voice chat
        await client.connect_voice_chat()
        
        # Set preferences
        await client.set_user_settings(VoiceType.ALLOY, LanguageCode.ENGLISH)

if __name__ == "__main__":
    asyncio.run(main())

Synchronous Usage

from spaceverseai import SpaceverseAIClientSync, EmotionType

with SpaceverseAIClientSync() as client:
    health = client.health_check()
    print(f"Server health: {health.status}")
    
    client.connect_voice_chat()
    client.set_client_emotion(EmotionType.HAPPY)

Run Examples

# Run interactive examples
spaceverseai-demo --interactive

# Run all examples
spaceverseai-demo

📋 Prerequisites

System Requirements

• Python: 3.8 or higher
• RAM: Minimum 4GB (8GB recommended)
• Storage: 2GB free space
• Network: Stable internet connection
• GPU: Optional (CUDA-compatible for faster emotion detection)

API Keys Required

• OpenAI API Key: Required for GPT-4o access
• Environment Variable: OPENAI_API_KEY

🏗️ Architecture

System Components

SpaceVerse AI
├── 🎤 Voice Processing
│   ├── Real-time audio streaming
│   ├── Audio format conversion (PCM16)
│   └── Sample rate resampling (24kHz)
├── 🎭 Emotion Detection
│   ├── Facial emotion analysis
│   ├── Real-time image processing
│   └── Emotion-to-response mapping
├── 🧠 AI Intelligence
│   ├── OpenAI GPT-4o integration
│   ├── User preference analysis
│   └── Threat detection system
├── 💾 Session Management
│   ├── Persistent session storage
│   ├── Metadata tracking
│   └── Event logging
└── 🌐 Web Interface
    ├── WebSocket connections
    ├── REST API endpoints
    └── Real-time updates

Technology Stack

• Backend: FastAPI (Python)
• AI Model: OpenAI GPT-4o Realtime
• Emotion Detection: Hugging Face Transformers
• Real-time Communication: WebSockets
• Data Processing: NumPy, PyTorch
• Image Processing: PIL (Pillow)
• Concurrency: asyncio, ThreadPoolExecutor

🎭 Emotion Detection System

Facial Emotion Processing

The system uses a pre-trained deep learning model to detect facial emotions in real-time:

# Emotion detection pipeline
processor = AutoImageProcessor.from_pretrained("dima806/facial_emotions_image_detection")
model = AutoModelForImageClassification.from_pretrained("dima806/facial_emotions_image_detection")

Emotion-to-Response Mapping

The AI adapts its communication style based on detected emotions:

Emotion	Tone	Speech Pattern	Approach
Happy/Joy	Joyful	Quick	High energy
Sad/Sadness	Empathetic	Soft	Gentle
Angry/Anger	Calm	Measured	Controlled
Fear/Fearful	Reassuring	Slow	Protective
Surprise	Engaging	Normal	Curious
Disgust	Understanding	Normal	Accepting
Neutral	Balanced	Normal	Moderate

🛡️ Safety and Threat Detection

Crisis Intervention System

The system includes advanced threat detection for user safety:

# Threat detection function for OpenAI
threat_detection_tool = {
    "type": "function",
    "name": "detect_threat_or_suicide",
    "description": "CRITICAL SAFETY FUNCTION: Detect suicidal intent, self-harm, or violence threats",
    "parameters": {
        "type": "object",
        "properties": {
            "contains_threat": {
                "type": "boolean",
                "description": "True if concerning content detected"
            }
        }
    }
}

Safety Features

• Real-time Monitoring: Continuous analysis of user messages
• Context Awareness: Distinguishes between academic discussion and genuine risk
• Emotional Context: Considers facial and voice emotions in threat assessment
• Crisis Resources: Provides immediate help when threats are detected
• False Positive Prevention: Careful analysis to avoid unnecessary alerts

🔧 Configuration

Environment Variables

Variable	Description	Default	Required
OPENAI_API_KEY	OpenAI API key for GPT-4o access	None	✅ Yes
LOG_LEVEL	Logging level (DEBUG, INFO, WARNING, ERROR)	INFO	❌ No
OPENAI_WS_URL	OpenAI WebSocket URL	GPT-4o Realtime	❌ No

Model Configuration

The application uses pre-trained models for emotion detection:

# Facial Emotion Detection Model
model = AutoModelForImageClassification.from_pretrained("dima806/facial_emotions_image_detection")
processor = AutoImageProcessor.from_pretrained("dima806/facial_emotions_image_detection")

Supported Emotions: happy, sad, angry, fearful, surprise, disgust, neutral

📊 API Endpoints

Health and Status

GET /health

Get application health status and connection information.

Response:

{
  "status": "healthy",
  "message": "OpenAI Realtime Voice Chat API is running",
  "connections": {
    "voice_websockets": 2,
    "image_websockets": 1,
    "openai_connections": 2,
    "total_sessions": 2
  }
}

Session Management

GET /api/sessions

List all active sessions with metadata.

GET /api/session/{session_id}/status

Get comprehensive session status and configuration.

GET /api/session/{session_id}/history

Get session information (OpenAI manages conversation history).

GET /api/session/{session_id}/events

Get OpenAI event IDs tracked for the session.

Client Configuration

GET /api/client/{client_id}/emotion

Get current detected emotion for a client.

POST /api/client/{client_id}/emotion

Manually set detected emotion and trigger session update.

GET /api/client/{client_id}/onboarding-data

Get current onboarding data for a client.

POST /api/client/{client_id}/onboarding-data

Set onboarding data and trigger session update.

🌐 Multi-Language Support

Supported Languages

Language	Code	Voice Support	Transcription
English	en	✅	✅
Spanish	es	✅	✅
French	fr	✅	✅
German	de	✅	✅
Italian	it	✅	✅
Portuguese	pt	✅	✅
Russian	ru	✅	✅
Japanese	ja	✅	✅
Korean	ko	✅	✅
Chinese	zh	✅	✅
Arabic	ar	✅	✅
Hindi	hi	✅	✅

🤝 Contributing

Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new features
5. Submit a pull request

Code Style

• Follow PEP 8 Python style guide
• Add comprehensive docstrings
• Include type hints
• Write unit tests for new features

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• OpenAI: For providing the GPT-4o Realtime API
• Hugging Face: For the emotion detection models
• FastAPI: For the excellent web framework
• Community: For feedback and contributions

📞 Support

Getting Help

• Documentation: This README and inline code comments
• Issues: Create GitHub issues for bugs or feature requests
• Discussions: Use GitHub Discussions for questions
• Email: Contact the development team

Common Issues

Connection Problems

• Check OpenAI API key is valid
• Verify internet connection
• Check firewall settings

Audio Issues

• Ensure microphone permissions
• Check audio format compatibility
• Verify sample rate settings

Emotion Detection Issues

• Ensure camera permissions
• Check image quality
• Verify model loading

---

SpaceVerse AI - Bringing intelligence and emotion to real-time conversations. 🚀