PC Voice Assistant

A local, privacy-focused voice assistant for Windows that runs entirely on your machine. Speak naturally to control your PC — open apps, set reminders, search the web, and more.

Features

• Wake Word Detection: Say "Alexa" or "Hey Computer" to activate (ML-based openWakeWord with energy fallback)
• Speech Recognition: Offline Vosk ASR engine — no internet required
• Natural Language Understanding: Rule-based intent matching with optional Ollama LLM integration
• Text-to-Speech: pyttsx3 (offline) or edge-tts (online, higher quality)
• Smart Reminders: Natural time expressions ("in 2 hours", "next Monday at 3pm")
• Fuzzy App Matching: "open chrome" matches "Google Chrome" automatically
• System Integration: Open apps, control volume, reminders, screenshots, and more
• System Tray: Runs in background with tray icon and context menu
• Rotating Logs: All activity logged to logs/assistant.log
• 133 Test Suite: Comprehensive unit tests for all modules

1. Project Overview

The PC Voice Assistant is a Python CLI application that provides hands-free voice control for Windows PCs. It listens for a wake word, records your voice, transcribes it to text, understands your intent, executes the action, and speaks a response.

Pipeline Flow:

Wake Word → Record Audio (VAD) → Transcribe → Parse Intent → Execute Action → Speak Response → Wait for Wake Word

The assistant is:

• Local/Offline: Core functionality works without internet
• Windows-Focused: Uses Windows-specific commands (PowerShell, Windows APIs)
• Privacy-First: All processing happens on your machine
• Extensible: Easy to add new intents and commands

2. Requirements

Python Version

• Python 3.10 or higher required (union type hints like Tuple[...] | None)

Operating System

• Windows (primary) — fully supported
• Linux/Mac: Wake word and TTS modules may work, but system commands (volume, shutdown, sleep) require Windows

Hardware

• Microphone required for voice input
• Any standard USB or built-in microphone works
• Audio output device required for TTS

External Tools (Optional)

3. Installation

Step 1: Clone the Repository

git clone <repository-url>
cd pc-assistant

Step 2: Install Python Dependencies

pip install -r requirements.txt

Step 3: Download and Install Vosk Model

1. Download from: https://alphacephei.com/vosk/models
2. Recommended model: vosk-model-small-en-us-0.15 (~45MB) or vosk-model-en-us-0.22 (~1.8GB)
3. Extract into the project root directory
4. The folder name should match asr.model_path in config.json (default: models/vosk-model-small-en-us-0.15)

pc-assistant/
├── models/
│   └── vosk-model-small-en-us-0.15/
│       ├── am/
│       ├── conf/
│       ├── graph/
│       └── ivector/
├── main.py
├── config.json
└── ...

Step 4: (Optional) Install and Start Ollama

For LLM-powered natural language understanding:

1. Install Ollama: https://ollama.ai
2. Pull a model:

   ollama pull llama3.2
   

3. Start Ollama server:

   ollama serve
   

4. Set nlu.use_ollama to true in config.json

Step 5: Run the Assistant

python main.py

Press Ctrl+C to exit gracefully.

4. Configuration (config.json)

All settings are in config.json. Never edit while the assistant is running.

wake_word Section

asr Section

nlu Section

tts Section

assistant Section

actions Section

reminders Section

ui Section

tray Section

logging Section

Reserved/Unused Config Keys

These keys exist in config.json but are not currently read by Python code. They are preserved for future use:

5. Voice Commands Reference

Greetings

Time & Date

Volume Control

Applications & Websites

Web Search

Reminders

Actions

Math

System Control

Conversation

6. Architecture Overview

Pipeline Flow

┌─────────────────────────────────────────────────────────────────┐
│                         MAIN LOOP                               │
├─────────────────────────────────────────────────────────────────┤
│  1. WAKE WORD DETECTION (wake_word.py)                         │
│     - Listen for wake word ("alexa", "hey computer")          │
│     - Uses openWakeWord ML model or energy threshold           │
│     - Blocking wait with VAD to detect speech start           │
├─────────────────────────────────────────────────────────────────┤
│  2. RECORD AUDIO (asr.py)                                     │
│     - Record until silence detected (VAD)                       │
│     - Configurable min/max duration                            │
│     - Audio enhancement: noise reduction, normalization         │
├─────────────────────────────────────────────────────────────────┤
│  3. TRANSCRIBE (asr.py)                                       │
│     - Vosk offline ASR → text                                 │
│     - Fallback to keyword detection if model unavailable       │
├─────────────────────────────────────────────────────────────────┤
│  4. PARSE INTENT (nlu.py)                                     │
│     - Keyword-based pattern matching                           │
│     - Optional Ollama LLM for unknown intents                  │
│     - Intent handlers: apps, reminders, volume, etc.           │
├─────────────────────────────────────────────────────────────────┤
│  5. EXECUTE ACTION                                             │
│     - Open apps via subprocess                                 │
│     - Set reminders (persisted to JSON)                        │
│     - Control volume via PowerShell                           │
│     - Screenshot via mss                                       │
├─────────────────────────────────────────────────────────────────┤
│  6. SPEAK RESPONSE (tts.py)                                   │
│     - pyttsx3 (offline Windows SAPI)                          │
│     - edge-tts (online, higher quality)                       │
│     - Falls back gracefully if primary fails                   │
└─────────────────────────────────────────────────────────────────┘

Module Responsibilities

7. Troubleshooting

Wake Word Never Triggers

Symptoms: Assistant doesn't respond to wake word.

Solutions:

1. Check microphone is working in Windows Sound settings
2. Verify wake_word.input_device in config.json is correct (or null for default)
3. Check wake_word.engine is set to "openwakeword" or "energy"
4. For openWakeWord: ensure it's installed (pip install openwakeword)
5. For energy fallback: lower wake_word.energy_fallback.threshold (e.g., 0.005)
6. Lower wake_word.confidence_threshold (e.g., 0.3)
7. Speak clearly and closer to the microphone

ASR Model Not Found Error

Symptoms: "Vosk model not found" error on startup.

Solutions:

1. Download model from https://alphacephei.com/vosk/models
2. Extract to project root, ensure folder structure:

   models/vosk-model-small-en-us-0.15/{am,conf,graph,ivector}/
   

3. Verify asr.model_path in config.json matches folder name
4. Check the path doesn't contain special characters

TTS Makes No Sound

Symptoms: Assistant speaks but you hear nothing.

Solutions:

1. Check tts.engine in config.json
   • pyttsx3: Uses Windows SAPI, check system volume
   • edge-tts: Requires internet connection
2. Run python -c "import tts; print(tts.get_engine_info())" to see active engine
3. Install pyttsx3 if missing: pip install pyttsx3
4. For edge-tts: pip install edge-tts
5. Check Windows audio device is not muted

Ollama Intents Not Working

Symptoms: Unknown commands return fallback instead of LLM response.

Solutions:

1. Confirm Ollama is running:

   ollama serve
   

2. Check model is installed:

   ollama list
   

3. Pull model if missing:

   ollama pull llama3.2
   

4. Verify nlu.use_ollama is true in config.json
5. Check nlu.ollama_base_url is http://localhost:11434
6. Test manually:

   curl http://localhost:11434/api/tags
   

Reminder Notification Not Showing

Symptoms: Reminder fires but no desktop notification.

Solutions:

1. Install plyer for desktop notifications:

   pip install plyer
   

2. Check Windows notification settings allow notifications from Python
3. Without plyer, reminders still work via TTS voice notification
4. Check reminders.enabled is true in config.json

App Open Command Not Working

Symptoms: "I couldn't find an app" response.

Solutions:

1. Check the app exists in actions.apps in config.json
2. Add the app with full path if needed:

   "notepad": "C:\\Windows\\notepad.exe"
   

3. Fuzzy matching handles partial names:
   • "open chrome" matches "google chrome"
   • "open vs code" matches "visual studio code"
4. For URLs, add to actions.websites:

   "youtube": "https://youtube.com"
   

8. Running Tests

python -m pytest tests/ -v

Expected Result: 133 tests, all passing.

Test Coverage

9. Adding New Intents

Step 1: Add App/Command to Config (if applicable)

For new apps or websites, add to config.json:

"actions": {
    "apps": {
        "myapp": "C:\\path\\to\\myapp.exe"
    }
}

Step 2: Create Intent Handler Class

In nlu.py, add a new handler class:

class NewIntentHandler(IntentHandler):
    def __init__(self):
        super().__init__("newintent", ["trigger word", "other trigger"])
    
    def handle(self, text: str) -> Optional[str]:
        # Your logic here
        return "Response message"

Step 3: Register the Handler

Add to INTENT_HANDLERS list in nlu.py:

INTENT_HANDLERS = [
    ...
    NewIntentHandler(),
]

Step 4: Add Tests

In tests/test_nlu.py:

def test_new_intent_handler(self):
    handler = nlu.NewIntentHandler()
    assert handler.match("trigger word")
    result = handler.handle("trigger word")
    assert "Response" in result

Step 5: Update README

Add your command to the Voice Commands Reference section.

Step 6: Run Tests

python -m pytest tests/ -v

10. Changelog

Version 1.1.0 — Current Development

New Features

• VAD Silence Detection: Recording now stops automatically when you stop speaking, instead of fixed duration. Configurable min/max duration and silence threshold.
• Fuzzy App Matching: "open chrome" automatically matches "google chrome" in config. Uses word overlap algorithm with configurable threshold.
• Extended Reminder Parsing: Natural time expressions now supported:
  • "in 2 hours"
  • "in 1 hour and 30 minutes"
  • "tomorrow at 9am"
  • "next monday at 3pm"
  • "in 90 seconds"
• Unknown Intent Fallback: Customizable response when no intent matches. Configurable via nlu.unknown_intent_response.

Reliability Improvements

• Config Caching: Config is now loaded once per module, not on every function call. Significant performance improvement.
• Atomic Reminder Writes: Uses temp file + os.replace() to prevent corruption if process is killed mid-write.
• Wake Word Race Condition Fix: Added _model_ready Event to properly synchronize background model loading.
• TTS MP3/WAV Fix: edge-tts generates MP3 but was being played as WAV. Now uses msedge.exe for correct playback.
• Path Quoting Fix: PowerShell commands now properly escape paths with spaces.

Security Improvements

• subprocess shell=True Removal: All subprocess calls now use list-based arguments, eliminating shell injection risk.

Testing

• 133 Test Suite: Comprehensive edge case coverage added:
  • ASR: mic failure, missing model, empty transcription, partial results
  • TTS: missing files, long strings, engine fallback
  • Wake word: timeout fallback, model loading errors
  • Main loop: error recovery from all stages
  • Reminders: concurrent writes, persistence, past time handling

License

MIT