unix-tutor 0.3.1

A terminal assistant that runs shell commands from natural language using a local LLM

unix-tutor

A terminal assistant that runs shell commands from natural language using a local LLM (Ollama). Works as a hybrid shell: type real commands and they run directly; prefix with llm-exec to translate natural language into a command first.

Install

pip install unix-tutor

Requires Ollama running locally with a model pulled:

ollama pull qwen2.5:7b
ollama serve

Usage

Terminal CLI

unix-tutor

Input	Behavior
ls -la	Runs directly as a shell command
llm-exec show all python files	LLM translates → shows command + explanation → [y/N] → runs
update-constraints	Add or remove blocked paths and soft constraint rules
exit	Quit

Desktop UI

unix-tutor-ui

Full agent-based chat interface with guardrails manager, command history, and file operations.

Guardrails

Add or remove constraints without restarting — type update-constraints at the prompt:

You> update-constraints

What would you like to do?
  1. Add blocked path    — prevent access to a specific file or directory
  2. Add soft constraint — a rule added to the agent's behavior
  3. Remove a constraint — delete an existing blocked path or soft constraint
Enter 1, 2, or 3:

Blocked paths are enforced at the tool layer — no command (direct or LLM-generated) can touch them, even if the path casing differs (macOS case-insensitive filesystem handled).

Soft constraints are injected into the agent's guidance in the desktop UI.

Constraints persist across sessions in ~/Library/Application Support/UnixAgent/guardrails.json.

Hard-blocked commands (always enforced, not configurable)

The following commands are always rejected regardless of input:

• Privilege escalation: sudo, su
• Network/remote: ssh, scp, curl, wget, nc, telnet
• Process/service control: kill, killall, shutdown, reboot, launchctl, systemctl
• Disk operations: mount, diskutil, fdisk, dd
• Shell spawns: bash, sh, zsh, fish

Destructive commands (rm, rmdir) require [y/N] confirmation before running.

Configuration

Settings are loaded from ~/Library/Application Support/UnixAgent/.env (auto-created on first run). Override by placing a .env in the directory where you run unix-tutor:

LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434
LLM_MODEL=qwen2.5:7b

Project Structure

src/unix_tutor_agent/
  cli.py          — hybrid terminal CLI (llm-exec prefix + direct shell)
  ui.py           — Tkinter desktop UI and agent backend
  agent.py        — agent construction and turn execution
  tools.py        — agent tools (run commands, file ops, history, guardrails)
  guardrails.py   — blocked paths and soft constraints (persisted to JSON)
  config.py       — settings loading from env
  session.py      — workspace-scoped shell session (cd state)
  db.py           — SQLite command history schema and logging
run_ui.py         — dev launcher for the desktop UI

Evaluation

The evaluation harness and dataset:

• Dataset: eval/scenarios.json
• Runner: eval/run_eval.py
• LLM candidate generator: eval/generate_llm_candidates.py
• Judge spot-check tool: eval/spot_check_judge.py

Run deterministic eval

python eval/run_eval.py --k 5

Run two configurations for comparison:

python eval/run_eval.py --k 5 --models "claude-haiku-4-5,claude-opus-4-6-v1"

Run one specific scenario:

python eval/run_eval.py --k 1 --scenario-id create_notes_dir

Pass@k Results

Model: claude-sonnet-4-6

• pass@1: 0.714 (10/14)
• pass@5: 0.786 (11/14)
• hard_safety_breach_rate: 0.000

Model: api-gpt-oss-120b

• pass@1: 0.500 (7/14)
• pass@5: 0.571 (8/14)
• hard_safety_breach_rate: 0.000

Hard-block guardrails are enforced at the tool layer regardless of model output — blocked commands are never executed.

Optional LLM-as-judge

python eval/run_eval.py --k 5 --use-llm-judge --judge-model "claude-opus-4-6-v1"

Spot-check judge verdicts

python eval/spot_check_judge.py --results eval/results/eval_results_<timestamp>.json --samples 5

Demo

https://youtu.be/cUaGNh2H8Jc