Ultron-1.0

Ultron-1.0 is a local CLI assistant that keeps the existing Ollama qwen2.5-coder:3b reasoning core and adds persistent SQLite conversation memory, a safe tool system, and an Ollama-powered planner-executor loop for repository tasks.

Requirements

• Python 3.12+
• uv
• Ollama running locally
• The qwen2.5-coder:3b model installed in Ollama

Setup

uv sync
cp .env.example .env
ollama pull qwen2.5-coder:3b
ollama pull nomic-embed-text

Edit .env if you want different paths or logging settings:

ULTRON_MODEL=qwen2.5-coder:3b
ULTRON_MEMORY_DB=data/ultron_memory.sqlite3
ULTRON_MEMORY_RETRIEVAL_LIMIT=6
ULTRON_LOG_LEVEL=INFO
ULTRON_LOG_FILE=logs/ultron.jsonl
ULTRON_PROJECT_ROOT=.
ULTRON_AGENT_MAX_STEPS=20
ULTRON_TASK_RETRY_BUDGET=3
ULTRON_TASK_QUALITY_THRESHOLD=0.84
ULTRON_ROLLBACK_ENABLED=true
ULTRON_SNAPSHOT_ROOT=data/snapshots
ULTRON_BENCHMARK_ROOT=data/benchmarks
ULTRON_DATASET_QUALITY_THRESHOLD=0.84
ULTRON_DATASET_EXPORT_DIR=data/dataset
ULTRON_DATASET_MIN_ACCEPTED=120
ULTRON_DATASET_MIN_CORE_CATEGORY=10
ULTRON_DATASET_MAX_DUPLICATE_RATE=0.05
ULTRON_TRAINING_BACKEND=auto
ULTRON_MLX_MODEL=mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit
ULTRON_MLX_ADAPTER_PATH=data/training-output/mlx-adapter
ULTRON_PLANNING_MODEL=qwen2.5-coder:3b
ULTRON_CODING_MODEL=qwen2.5-coder:3b
ULTRON_CRITICISM_MODEL=qwen2.5-coder:3b
ULTRON_VERIFICATION_MODEL=qwen2.5-coder:3b
ULTRON_EMBEDDING_MODEL=nomic-embed-text
ULTRON_REPOSITORY_INDEX=data/repository_index.json
ULTRON_EXTERNAL_NETWORK=false

Do not change ULTRON_MODEL if you want to keep the default qwen2.5-coder:3b core.

Run

Start Ollama:

ollama serve

Run chat:

uv run python main.py chat

Type exit or quit to leave the chat.

The default command is also chat, so this still works:

uv run python main.py

CLI Commands

Inspect registered tools and typed schemas:

uv run python main.py inspect

Run a planned repository task:

uv run python main.py run-task "list files"
uv run python main.py run-task "search SQLite memory"
uv run python main.py run-task "test project"
uv run python main.py run-task "add a pytest for memory retrieval" --dry-run

Resume or inspect persisted tasks:

uv run python main.py tasks
uv run python main.py resume 1
uv run python main.py task-report 1
uv run python main.py task-diff 1
uv run python main.py snapshots
uv run python main.py restore task-1-20260101000000000000

Index and inspect repository architecture:

uv run python main.py index
uv run python main.py architecture
uv run python main.py explain-file ultron/core/planner.py
uv run python main.py impact ultron/core/planner.py

Run learning and inspect lessons:

uv run python main.py learn
uv run python main.py lessons
uv run python main.py review-lessons
uv run python main.py forget-lesson 1
uv run python main.py lesson-experiment 1
uv run python main.py lesson-report 1
uv run python main.py lesson-performance
uv run python main.py lesson-promote 1
uv run python main.py lesson-disable 1
uv run python main.py lesson-rollback 1
uv run python main.py self-learning-benchmark
uv run python main.py self-learning-report

Run benchmarks and metrics:

uv run python main.py benchmark
uv run python main.py benchmark-coding
uv run python main.py benchmark-report
uv run python main.py benchmark-history
uv run python main.py compare-benchmarks
uv run python main.py benchmark-advanced
uv run python main.py evaluate-task 1
uv run python main.py metrics

Prepare training data:

uv run python main.py dataset-review
uv run python main.py dataset-generate --count 20 --categories feature,bugfix,refactor,test,review
uv run python main.py dataset-auto-review
uv run python main.py dataset
uv run python main.py dataset-approve 1
uv run python main.py dataset-export --format chat
uv run python main.py dataset-export --format instruction
uv run python main.py dataset-export --format code-edit
uv run python main.py dataset-export --format preference
uv run python main.py dataset-stats
uv run python main.py dataset-coverage
uv run python main.py dataset-leakage-check
uv run python main.py dataset-quality-report
uv run python main.py mlx-dataset-export
uv run python main.py training-readiness
uv run python main.py prepare-training
uv run python main.py training-preflight --backend mlx
uv run python main.py training-command --backend mlx --dry-run
uv run python main.py training-preflight --dataset data/dataset/train.instruction.jsonl --output-dir data/training-output
uv run python main.py training-command --dataset data/dataset/train.instruction.jsonl --base-model Qwen/Qwen2.5-Coder-1.5B --output-dir data/training-output
uv run python main.py model-candidates
uv run python main.py model-candidate-report candidate-id
uv run python main.py replacement-benchmark
uv run python main.py replacement-report
uv run python main.py evaluate-ultron
uv run python main.py package-ultron
uv run python main.py register-ultron
uv run python main.py ultron-health
uv run python main.py promote-model --role critic
uv run python main.py promotion-status
uv run python main.py rollback-model --role critic
uv run python main.py release-readiness
uv run python main.py experiments
uv run python main.py shadow-evaluate "add a small feature"
uv run python main.py learning-report

Install optional training dependencies only when preparing to fine-tune:

uv sync --group training

On Apple Silicon, install MLX-LM only when preparing a local MLX fine-tune:

uv sync --group mlx

Inspect memory:

uv run python main.py memory --kind messages
uv run python main.py memory --kind tool-calls
uv run python main.py memory --kind lessons --query pytest

Show current status:

uv run python main.py status

Memory

Ultron stores every user message and assistant response in SQLite at the path configured by ULTRON_MEMORY_DB. Before each response, it retrieves relevant past messages and injects them as context for Ollama.

The same SQLite database stores tool calls, tool results, failures, task checkpoints, and trusted lessons for future retrieval before similar planned tasks. Unverified model statements are not stored as trusted lessons.

Semantic memory uses local Ollama embeddings through nomic-embed-text by default. If embeddings are unavailable, Ultron falls back to deterministic local hash embeddings and continues operating.

Agent Loop

run-task uses qwen2.5-coder:3b through Ollama to produce strict JSON plans validated by Pydantic. Each plan includes:

• goal
• steps
• tool name
• tool arguments
• expected result
• risk level
• approval required

If the model returns invalid JSON or an invalid schema, Ultron retries and then falls back to a safe read-only plan. During execution it inspects repository context, retrieves relevant memories and lessons, executes allowed tools, observes output, revises plans after safe failures, and stops when complete or safely blocked.

After edits, Ultron automatically runs:

uv run pytest
uv run ruff check .
uv run ruff format --check .

It then inspects Git diff and runs a separate critic prompt to decide whether the task is complete, should retry, or is blocked.

Dry-run mode plans and previews actions without modifying files:

uv run python main.py run-task "create docs/notes.md with a short note" --dry-run

Tools And Safety

Registered tools are grouped as:

• Read-only repository tools: list files, read file, search code, inspect Git status.
• Controlled write tools: create file, edit file, apply patch, and delete file only with explicit approval.
• Terminal tool: runs inside ULTRON_PROJECT_ROOT, blocks dangerous commands, supports timeouts, and captures stdout, stderr, and exit code.
• Git tools: diff, status, and commit preparation. Push is never automatic.

Approval gates are enforced for file deletion, broad rewrites, package installation, commands outside the project, and Git commit/push operations.

The planner-executor loop creates a short plan, chooses tools, executes each step, inspects results, stops on approval gates or unsafe failures, and records a self-review.

Write operations include previews in task output. File deletion, broad rewrites, package installation, commands outside the project, and commit preparation ask for explicit approval. Git push is blocked automatically.

Patch application validates unified-diff structure and project-local paths, backs up changed files, and restores files automatically if patch application fails.

Logging

Logs are structured JSON lines written to stderr and, by default, logs/ultron.jsonl.

Ultron-1.1

Ultron-1.1 layers an evidence-first foundation on top of the working v1.0 core, in the ultron/v1_1/ package. It does not modify or remove any v1.0 component, command, or data: it adds new SQLite tables with CREATE TABLE IF NOT EXISTS and keeps the v1.0 SQLite database fully intact.

Every measurement carries an explicit provenance label so live model execution, deterministic fixture execution, and static validation are never confused:

• live — genuine live model execution.
• fixture — deterministic reference-solution execution (validates the harness).
• static — static validation (lint, parse, policy checks).
• inferred / unavailable — derived or not measurable in this environment.

See V1_1.md for full methodology, the lesson lifecycle, recovery and rollback semantics, dataset provenance, the security model, and honest limitations.

New v1.1 commands

# Live task execution (v1.1 orchestrator: MLX default, Qwen fallback, durable
# exact-once ledger, acceptance enforcement, evidence + dataset eligibility)
uv run python main.py run-task "fix the failing test in module X"
uv run python main.py run-task "..." --solver qwen      # force the Qwen fallback
uv run python main.py run-task "..." --legacy           # v1.0 Ollama planner path
uv run python main.py resume-task <task_id>             # exact-once durable resume
uv run python main.py task-status <task_id>
uv run python main.py task-evidence <task_id>
uv run python main.py task-history <task_id>

# Real execution-based benchmarks (sandbox copies, real pytest/node, real exit codes)
uv run python main.py benchmark-real --solver reference   # fixture harness validation
uv run python main.py benchmark-real --solver ultron      # LIVE Ultron MLX (provenance=live)
uv run python main.py benchmark-real --solver qwen        # LIVE Qwen fallback
uv run python main.py benchmark-real --task bug-001 --solver ultron
uv run python main.py benchmark-real-list
uv run python main.py benchmark-real-report --solver ultron
uv run python main.py compare-versions

# Non-benchmark real-task collection (isolated copies, real verification)
uv run python main.py real-task-list
uv run python main.py real-task-run
uv run python main.py real-task-report
uv run python main.py real-task-approve <task_id>
uv run python main.py real-task-reject <task_id>

# Real v1.1 dataset (built only from approved verified real tasks; never trains)
uv run python main.py dataset-v1.1-provenance
uv run python main.py dataset-v1.1-exclusions

# Verified self-learning v2 (real control vs treatment experiments)
uv run python main.py learning-evaluate
uv run python main.py learning-regressions
uv run python main.py learning-prune
uv run python main.py lesson-list
uv run python main.py lesson-history lesson-remove-guards
uv run python main.py lesson-ab-test lesson-duplicate-constant
uv run python main.py lesson-disable lesson-duplicate-constant
uv run python main.py lesson-restore lesson-duplicate-constant

# Evidence-based model routing v2
uv run python main.py routing-status
uv run python main.py routing-history
uv run python main.py routing-evaluate
uv run python main.py routing-promote --prefer ultron-1.1 --min-success 0.6
uv run python main.py routing-rollback

# Versioned real-task dataset pipeline (separate from v1.0; never trains automatically)
uv run python main.py dataset-v1.1-build
uv run python main.py dataset-v1.1-report
uv run python main.py dataset-v1.1-leakage
uv run python main.py prepare-v1.1-training
uv run python main.py v1.1-training-readiness

# Production diagnostics (doctor --repair is conservative and never deletes data)
uv run python main.py doctor
uv run python main.py doctor --repair
uv run python main.py system-health
uv run python main.py diagnostic-report

# Security regression audit
uv run python main.py security-audit
uv run python main.py security-audit-report

# Repository intelligence v2 (multi-language, evidence-backed)
uv run python main.py repo-intel-index
uv run python main.py repo-intel-impact ultron/v1_1/store.py
uv run python main.py repo-intel-architecture

# Readiness and release gating (release-v1.1 is report-only and never mutates)
uv run python main.py v1.1-report
uv run python main.py v1.1-readiness
uv run python main.py release-v1.1

All v1.1 commands emit machine-readable JSON.

Ultron-2.1 — Verified Operator Runtime

Ultron-2.1 adds one canonical runtime and connects it to real entrypoints:

request -> classify -> plan -> policy -> execute -> observe -> evidence
        -> verify -> bounded correction -> verified response -> verified memory

The point is that completed and verified are different words, and Ultron means both literally. A run that gathered truncated evidence says so; a claim that cites a failed test run is marked contradicted, not summarised as success; a claim citing an evidence id that does not exist is unsupported.

# Ask something. Each stage prints as it happens.
uv run python main.py operator run "what runtime states does lifecycle.py define?"

# Plan and policy-check everything, execute nothing.
uv run python main.py operator run "..." --dry-run

# Authorise an approval-risk tool for this run only.
uv run python main.py operator run "..." --approve report.write

# Inspect, replay, and re-verify a past run. Replay executes nothing.
uv run python main.py operator inspect RUN_ID
uv run python main.py operator replay RUN_ID
uv run python main.py operator verify RUN_ID
uv run python main.py operator resume RUN_ID

# The boundaries, stated explicitly.
uv run python main.py operator capabilities
uv run python main.py operator policy
uv run python main.py operator memory

# Chat on the verified runtime. Legacy chat is unchanged and still the default.
uv run python main.py operator chat
uv run python main.py chat --verified

Safety boundaries: policy is deny-by-default and every collection is an allowlist, so forgetting to configure something denies it. Tool arguments are argv-only with shell metacharacters rejected. Network access is not granted; network.nmap and network.capture are registered but disabled, and network.metasploit is hard-denied — the engine refuses it before it considers approvals, so no runtime approval can reach it.

Offline PCAP analysis is supported through tshark with -r and -n, so analysis can neither capture nor trigger DNS. TLS output is labelled handshake metadata; nothing claims decrypted contents.

See docs/ULTRON_2_1_OPERATOR_RUNBOOK.md to drive it, docs/ULTRON_2_1_ARCHITECTURE.md for the design, and docs/ULTRON_2_1_RELEASE_PLAN.md for measured gate results and honest limitations.

Tests

uv run pytest
uv run ruff check .
uv run ruff format --check .