name: socratic-watchdog aliases: [socratic-watchdog] tags: [project, tool] stage: playground

🧠 Socratic Watchdog

A Socratic TTS coding assistant that watches your Jupyter notebook cells, analyzes them through the lens of the Socratic method, and speaks guiding questions when you go off-track. Stays silent when your code is correct — and celebrates with confetti when you get it right.

"I cannot teach anybody anything, I can only make them think." — Socrates

▶️ Try it live: open the demo in Google Colab — no install, runs in your browser.

🧪 Test every feature yourself: open the manual test notebook in Colab — one cell per feature, each with an "Expected" note so you can confirm it works (includes a Colab install cell).

Features

Quick start

%load_ext socratic_watchdog
%socratic_task Write a function that calculates Fibonacci numbers

%%socratic
def fib(n):
    return fib(n-1) + fib(n-2)  # missing base case!

Socrates will (verbally) ask something like:

"I see your function calls itself — what condition would stop this recursion?"

When you fix it and it's correct, you'll get confetti and something like:

"Well done! You are thinking clearly."

Installation

pip install socratic-watchdog

The package has zero required dependencies beyond ipython. TTS backends and LLM access are configured via optional extras or environment variables:

# Optional: install a specific TTS backend
pip install socratic-watchdog[edge-tts]   # Microsoft neural voices (cloud, free)
pip install socratic-watchdog[kokoro]     # Local neural TTS (82M model, needs torch)

No extra pip install needed for espeak-ng — the package calls the system espeak-ng binary directly if it's on your PATH. Install it via apt install espeak-ng (Linux) or brew install espeak-ng (macOS).

LLM access

The package calls an LLM over a plain HTTPS POST to any OpenAI-compatible chat-completions endpoint (DeepSeek by default). It tries these API-key env vars in order, first match wins: SOCRATIC_LLM_API_KEY → DEEPSEEK_API_KEY → OPENAI_API_KEY. If none are set, Socrates runs in test-only mode (the #Test cases fast path still works; no questions are generated).

A Hermes agent backend is planned but not yet implemented — see HERMES_INTEGRATION.md for the design.

How it works

┌─────────────┐    ┌──────────────┐    ┌───────────┐    ┌──────────┐
│ Cell runs   │ →  │ Capture      │ →  │ LLM with  │ →  │ On track?│
│ (source +   │    │ source code  │    │ Socrates  │    │ → SILENT │
│  traceback) │    │ + error      │    │ persona   │    │ Off track│
└─────────────┘    └──────────────┘    └───────────┘    │ → TTS Q  │
                                                        └──────────┘

When test cases are set (via a #Test cases cell below or %socratic_generate_tests), there's a fast path: if the code passes all tests, the LLM is skipped entirely — instant silent + confetti.

The Socrates persona instructs the LLM to:

1. Never give direct answers or show corrected code
2. Ask exactly one guiding question
3. Reference something specific in the student's code
4. Stay completely silent when correct

Commands

Test cases

Author-written test cases let Socrates check correctness deterministically — no LLM guesswork. Put them in a code cell directly below your %%socratic cell, marked with #Test cases (also accepted: #Tests, #test_cases). The watchdog scans that cell, runs its assert lines as hidden tests, and caches them on disk keyed by task.

Cell-below tests are a track-mode feature — use %socratic_task auto, not an explicit task. The watchdog finds the cell below only while auto-detecting the task from the markdown above (it locates your cell in the saved notebook, then reads the cell after it). If you set an explicit %socratic_task, that scan is skipped and the tests come from the cache or the LLM instead. Save the notebook so the on-disk scan can see the cells.

%load_ext socratic_watchdog
%socratic_task auto        # track mode — read the task from the markdown above

**Task:** Write a function `is_even(n)` that returns True for even numbers.

%%socratic
def is_even(n):
    return n % 2 == 0

#Test cases
assert is_even(0) == True
assert is_even(1) == False
assert is_even(42) == True
assert is_even(-7) == False

• Pass all tests → instant silent + confetti, LLM skipped entirely.
• Fail with an LLM available → the failures are fed to the LLM for a sharper question.
• Fail with no LLM available → reported as a plain failure (never false praise).

The #Test cases cell runs like any other cell, so students see it. To keep tests hidden, use LLM-generated tests instead:

%socratic_task Write a function that reverses a string
%socratic_generate_tests  # LLM writes 4-6 assert statements, cached on disk

Audio toggle

%socratic_audio off   # TTS disabled
%socratic_audio on    # TTS re-enabled

Model switching

%socratic_model         # show numbered list of providers
%socratic_model 2       # pick #2 from the list
%socratic_model gpt-4o  # custom model name

Debug mode

%socratic_debug on   # shows test counts, LLM model+timing, TTS backend+timing
%socratic_debug off

Style toggle

%socratic_style brief    # direct questions, no preambles
%socratic_style verbose  # patient, playful mentor tone (default)

Exploration mode

%socratic_explore on    # Socrates encourages free experimentation
%socratic_explore off   # back to task-driven mode

Auto-generated tests

%socratic_auto_tests on   # auto-gen hidden tests on every %socratic_task
%socratic_auto_tests off
%socratic_clear_cache     # clear cached tests (useful for demos)
%socratic_cache           # inspect the generated-tests cache

Configuration

TTS

LLM

Other

Architecture

socratic_watchdog/
├── _core.py        # Core engine (no IPython deps — works anywhere)
│   ├── SocraticWatchdog.analyze()       # prompt → LLM → question/silence
│   ├── SocraticWatchdog.speak()         # text → TTS (kokoro/edge-tts/espeak) → Audio
│   ├── SocraticWatchdog.generate_tests() # LLM → test cases (disk-cached)
│   └── _call_llm()                      # OpenAI-compatible HTTPS POST (DeepSeek default)
├── magics.py       # IPython magics + post-run hook
│   ├── %%socratic, %socratic_task, %socratic_generate_tests, etc.
│   ├── _post_run_cell_hook              # auto-watch mode
│   ├── confetti animation               # canvas confetti on correct answers
│   └── 90+ Socratic praise phrases      # random praise on correct answers
└── __init__.py     # %load_ext entry point

Seeing the notebook is the part that surprises people. A kernel doesn't know about the document driving it, so _get_notebook_cells() tries three sources and takes the first that answers: the Colab frontend → jupyter-mcp-cli (JupyterHub / DIVE) → globbing the .ipynb off disk. The layer is read-only; nothing here writes cells back. ARCHITECTURE.md has the table and the failure modes.

Alongside the package, tutorial/ holds the CS1302 teaching material — a standalone, minimal version of those primitives built as spec-plus-failing-test so a small model can implement them one at a time. Not importable from an install, not shipped in the sdist.

Classroom reports

Every analysed cell appends one JSON line to ~/.hermes/socratic-sessions/<date>.jsonl (task, code, verdict, question, attempt count). A nightly script turns a day of those into a professor report:

python scripts/nightly_report.py                 # yesterday, to stdout
python scripts/nightly_report.py --date 2026-07-09 --out report.md

It reports per-student pass rates, flags stuck loops (≥4 attempts, never passed) and offline-LLM cells, and — if an API key is set — runs an answer-leak review that flags questions where Socrates gave away the fix. See HERMES_INTEGRATION.md for the cron setup and how a Hermes agent takes this further.

Privacy: logging is on by default and records student code. Set SOCRATIC_SESSION_LOG=off to disable, and SOCRATIC_STUDENT to label who's at the keyboard. Tell your students their work is logged.

Contributing

New here? Read ARCHITECTURE.md first — it maps every file, traces how one cell flows through the system, and defines the terms the code assumes. Roadmap lives in BACKLOG.md; the planned Hermes agent backend is designed in HERMES_INTEGRATION.md.

The golden rule: logic goes in _core.py (no Jupyter dependency, unit-tested); presentation goes in magics.py. Add a test in tests/test_socratic_watchdog_core.py, run pytest -q, and if you change behavior, update this README in the same commit.

License

MIT

Related

• edge-tts — free TTS engine
• Kokoro — local neural TTS (82M model)
• Hermes Agent — the agent framework