sangam

An LLM harness — a runtime and orchestration framework for coding agents backed by large language models.

sangam is being built as an LLM harness: a runtime that hosts LLM-driven coding agents, gives them isolated places to run code, and a typed set of tools to call. The design rests on four pieces —

• Runtime-agnostic execution — a single Executor interface (RunSpec in, RunResult out) with pluggable backends (Python venv, Node/nvm, Docker/Podman, subprocess) so the agent never depends on a specific runtime.
• Typed tool registry — each tool (read/write/edit files, run code, type-check, run tests, search) exposes a JSON schema, letting the LLM act through function-calling.
• Model client — an LLM abstraction (complete / chat / tools) that can talk to any provider: any OpenAI-compatible endpoint, or local models (Ollama / LM Studio). When no API key is set it falls back to a local model automatically.
• Agent loop — plan → act (call a tool) → observe (tests/lint/output) → reflect, with conversation memory and resumable state.

It started as a mypy-focused natural-language CLI, and those capabilities — mypy type analysis, file operations, and resumable task history — stay on as a built-in toolset and a working surface. Full architecture in docs/ARCHITECTURE.md.

Status

The harness is under active construction: the core agent loop, multi-runtime executors, tool registry, model client, session store, headless HTTP API server, and VS Code extension are in place (Phases 1–4 complete). The local webapp and desktop app surfaces are the remaining roadmap. See todo.md for the roadmap.

Available now

• Core engine split from the CLI surface (core/engine.py) — usable as a library.
• Executor protocol + RunSpec / RunResult runtime contracts (core/runtime/base.py).
• Multi-runtime executors — pluggable backends behind the single Executor interface:
  • VenvExecutor (core/runtime/venv.py) — creates an isolated venv, installs deps via pip, runs Python, captures stdout/stderr/exit.
  • NvmExecutor (core/runtime/nvm.py) — runs JS/React via nvm + npm (nvm use, npm install, run/build).
  • ShellExecutor (core/runtime/shell.py) — runs native bash/zsh scripts and one-off commands, auto-detecting the user's shell.
  • ContainerExecutor (core/runtime/container.py) — runs arbitrary code in Docker or Podman (image pull, run).
  • SubprocessExecutor (core/runtime/subprocess.py) — plain subprocess fallback backend.
• Runtime auto-detection (core/runtime/detect.py) — maps a runtime name (python/venv, node/js/react/npm, shell/bash/zsh, container/docker/podman, subprocess/direct) to the matching backend.
• Environment registry (core/runtime/registry.py) — records created venvs and npm projects in ~/.sangam/environments.json so they are reused across runs instead of recreated.
• Sandbox Manager (core/sandbox.py) — SandboxPolicy + SandboxedExecutor wrapping any backend with filesystem scope (path-level validation, symlink resolution), wall-clock timeouts (default/max clamping), network isolation (proxy-env clearing), and POSIX resource limits (cpu_time, fsize, nofile) via resource_limit_preexec.
• Agent loop — Agent (core/agent.py) orchestrating Planner (core/planner.py), Memory (core/memory.py), the ToolRegistry, and an optional ModelClient in a plan → act → observe → reflect loop. Runs in a deterministic mode when no model client is configured.
• Model client (core/model/) — ModelClient protocol with OpenAIModelClient (any OpenAI-compatible endpoint) and LocalModelClient (local providers such as Ollama / LM Studio); credentials read from env/config. build_model_client(config) picks the backend — the entry selected from the model catalog when one is configured, otherwise OpenAI when an API key is set, otherwise a local model (auto-detecting an available model from the running server when MODEL_NAME is unset).
• Model catalog (core/model/catalog.py) — ~/.sangam/models.yaml is the source of truth for which models sangam can use, following Continue's config.yaml models shape (name, provider, model, optional apiBase / apiKey / roles / capabilities / defaultCompletionOptions / requestOptions). The active chat/agent model is the file's default entry, else the first entry with role chat, else the first entry. sangam models lists the configured entries merged with the ids advertised by probed local runtimes (Ollama, LM Studio, and every configured apiBase) via /v1/models. sangam never installs a model runtime.
• Typed tool registry (core/tools/) — read_file, write_file, edit_file, list_files, search, run_python, run_js, run_container, run_shell, install_deps, type_check, run_tests, suggest_fixes, each with a JSON schema for LLM function-calling.
• Session store (core/session_store.py) — SQLite-backed sessions, messages, model/token usage, and FTS5 search (~/.sangam/sessions.db by default).
• CLI subcommands — sangam agent "<goal>", sangam models, sangam exec --runtime <rt> <command>, and sangam serve (headless HTTP API server), plus the legacy single-task positional, REPL, and batch mode.
• Headless HTTP API server (sangam/server/, Flask) — exposes the agent, tools, runtimes, and session store over local HTTP with SSE streaming; the single backend for the VS Code extension, local webapp, and desktop app.
• install.sh — pulls sangam from GitHub main, installs it into a venv at ~/.sangam/venv, and runs the headless server. Before starting it probes for a local model runtime (Ollama / LM Studio); if none is reachable it errors out with "no model available/selected" rather than installing one.
• mypy integration wired through the Executor backends — analyze a file or a whole directory, JSON output parsing, and fix suggestions. Defaults to SubprocessExecutor (sys.executable -m mypy); opt-in use_venv=True → VenvExecutor with mypy installed; custom backend via executor=.
• Regex-based task engine — natural-language commands mapped to file operations and mypy analysis.
• File operations — read, write, append, delete, list, and extract functions/classes/imports.
• JSONL task history with search.
• Interactive REPL and batch mode.

Roadmap

• VS Code extension polish — chat panel, diagnostics, inline actions (extension scaffolded and installable; UI activation pending a reload).
• Local webapp — a single self-contained index.html (no build step) served on localhost via Python (sangam web), talking to the headless server over HTTP.
• Cross-platform desktop app — React UI over the Python core behind a native shell, packaged for macOS (.app/.dmg) and Windows (.exe/.msi); reuses the headless server as its backend.
• Hardening — doctor capability probing, resumable history, runtime/model/ sandbox config, end-to-end sandbox limits, CI-ready non-interactive mode.

Local Webapp — Roadmap

A single-page webapp is a zero-install surface for the same harness: one self-contained index.html (inline HTML + CSS + JS, no build step) served on localhost by a small Python static server and talking to the core over HTTP. It reuses the headless HTTPS API server from Phase 4 as its backend — another thin surface over the same core, not a separate engine.

• webapp/index.html — the app shell (chat panel, file tree, diagnostics, agent-step viewer, command palette, status bar) with an inline fetch-based client to the core serve endpoint.
• sangam web — starts the headless server and serves index.html on localhost via Python's stdlib http.server (no extra deps); opens the default browser.

Full task breakdown in todo.md, Phase 6.

Desktop App — Roadmap

A cross-platform native desktop app (macOS + Windows) is a target surface for the same harness. It keeps the existing Python core as the engine and adds a React UI on top — the same "core is a library, surfaces are thin" split used for the CLI and the VS Code extension:

• Python core — the sangam package, frozen into a standalone per-OS binary and run headless (sangam serve, the same HTTPS API server the VS Code extension talks to). All agent/tool/runtime logic stays in Python.
• React UI — a TypeScript + React frontend (chat panel, file tree, diagnostics, agent-step viewer, command palette) that speaks HTTPS to the core over localhost (optionally with TLS/cert configuration) — the same wire protocol as the VS Code extension.
• Native shell — a desktop wrapper that bundles the React build and spawns the frozen Python core as a sidecar process. Ships a native .app/.dmg on macOS and .exe/.msi on Windows (Tauri preferred for the OS-webview native feel and small binary; Electron as a fallback).

Planned packaging steps (full task breakdown in todo.md, Phase 7):

1. React UI — scaffold desktop-ui/ (Vite + TypeScript + React); build the app shell and a typed HTTPS client to the core server.
2. Native shell — scaffold desktop-app/ (Tauri or Electron); configure it to bundle and spawn the frozen Python core sidecar; wire frontend↔core IPC over localhost HTTPS.
3. Python core packaging — freeze the core with PyInstaller/Nuitka per OS (including mypy + deps); confirm the headless serve entry point runs the agent loop end-to-end.
4. macOS build — universal .app (arm64 + x86_64), .dmg installer, Developer ID code signing, Apple notarization + stapling, clean-launch test.
5. Windows build — .exe + .msi installer (WiX/Tauri), WebView2 runtime handling, code signing, clean-launch test.
6. Release CI — GitHub Actions matrix (macOS arm64/x86_64, Windows x64): build → sign → notarize (macOS) → package → attach to GitHub Releases.
7. Updates & polish — auto-update, native menus/tray, first-run onboarding (LLM creds + runtime detection), native window-state persistence.

This builds on the headless HTTPS API server from Phase 4 — the desktop app is another thin surface over the same server, not a separate backend.

Installation

pip install -e .

Requires Python >= 3.9 and mypy>=0.900 (installed automatically).

Headless server install (install.sh)

To install sangam and run it as a headless HTTP API server (the backend for the VS Code extension, local webapp, and desktop app), use install.sh:

./install.sh                 # install (or update) and run the server
./install.sh --install-only  # install/update but do not start the server
./install.sh --port 8765     # bind a specific port (default 8765)

install.sh pulls the source from GitHub main, creates a venv at ~/.sangam/venv, installs the package, and runs sangam serve. The server needs a model to drive the agent, so before starting it install.sh probes for a local model runtime:

• It queries the OpenAI-compatible /v1/models endpoint of Ollama (127.0.0.1:11434) and then LM Studio (127.0.0.1:1234).
• If a reachable endpoint advertises a model, the server is started using that local model.
• sangam never installs a model runtime. If no Ollama or LM Studio endpoint is reachable, install.sh errors out with "no model available/selected" and exits without starting the server.

Override the probe with environment variables:

SANGAM_MODEL_ENDPOINT=http://127.0.0.1:12345 SANGAM_MODEL_NAME=bonsai-27b-mlx ./install.sh

--install-only installs without requiring a model.

Usage

CLI

# Run a single task
sangam "read main.py"

# Interactive mode
sangam --interactive

# Batch mode (from a file)
sangam --batch tasks.txt

# Verbose output
sangam --verbose "type check src/"

# Run the coding agent on a goal
sangam agent "fix the type errors in src/ and run the tests"

# List the models sangam can use (configured + discovered)
sangam models
sangam models --no-probe          # configured models only (no network)
sangam models --json              # machine-readable listing
sangam models --select fast-local # show the client for a named entry

# Run a command in a chosen runtime
sangam exec --runtime python "print('hello')"
sangam exec --runtime node "console.log('hello')"
sangam exec --runtime shell "ls -la"

Model catalog (~/.sangam/models.yaml)

~/.sangam/models.yaml is the source of truth for which models sangam can use. It follows Continue's config.yaml models shape:

default: fast-local          # optional: the active chat/agent model

models:
  - name: fast-local
    provider: ollama
    model: qwen3:0.6b
    apiBase: http://127.0.0.1:11434/v1
    roles: [chat, agent]
    capabilities: [tool_use]
    defaultCompletionOptions:
      temperature: 0.2
      maxTokens: 2048
    requestOptions:
      timeout: 60  # seconds

  - name: remote-gpt
    provider: openai
    model: gpt-4o-mini
    apiKey: $OPENAI_API_KEY   # literal, $VAR, or ${VAR}
    roles: [chat]

Selection order: the file's default entry, else the first entry with role chat, else the first entry. sangam models merges those configured entries with the model ids advertised by probed OpenAI-compatible runtimes (Ollama on 127.0.0.1:11434, LM Studio on 127.0.0.1:1234, and every configured apiBase) via /v1/models. Probing is read-only and best-effort — sangam never installs a model runtime. Override the catalog path with SANGAM_MODELS_FILE or sangam models --catalog <path>.

requestOptions carries transport-level settings for the entry's HTTP calls. timeout (seconds) is honoured on every chat-completions request and overrides the SANGAM_LLM_TIMEOUT environment variable (which defaults the timeout to 120 s). Invalid or non-positive values are ignored.

As a library

from sangam import SangamCLI

cli = SangamCLI()
result = cli.execute_task("type check main.py")
print(result)

The runtime contracts are importable for building backends:

from sangam.core import Executor, RunSpec, RunResult

The agent harness is importable too:

from sangam.core.agent import Agent
from sangam.core.planner import Planner
from sangam.core.tools import ToolRegistry
from sangam.core.session_store import SessionStore
from sangam.core.model import (
    ModelClient,
    OpenAIModelClient,
    LocalModelClient,
    build_model_client,
)

Project Structure

sangam/
├── __init__.py            # Package exports
├── cli/
│   ├── main.py            # CLI surface (argparse, REPL, batch mode)
│   └── commands/
│       ├── agent.py       # sangam agent "<goal>"
│       ├── models.py      # sangam models (list/select from models.yaml)
│       ├── exec.py        # sangam exec --runtime <rt> <command>
│       └── serve.py       # sangam serve (headless HTTP API server)
├── core/
│   ├── engine.py          # Core engine (config + logger + task engine wiring)
│   ├── agent.py           # Agent orchestrator (plan → act → observe → reflect)
│   ├── planner.py         # Goal → step plan
│   ├── memory.py          # Conversation history + working-set
│   ├── session_store.py   # SQLite sessions/messages/usage + FTS5 search
│   ├── sandbox.py         # SandboxPolicy, SandboxedExecutor, resource limits
│   ├── model/
│   │   ├── __init__.py    # exports + build_model_client(config) factory
│   │   ├── client.py      # ModelClient protocol + data contracts
│   │   ├── catalog.py     # ~/.sangam/models.yaml catalog (load/select/probe)
│   │   ├── model.py       # OpenAIModelClient (OpenAI-compatible)
│   │   └── local.py       # LocalModelClient (local providers, e.g. Ollama/LM Studio)
│   ├── tools/
│   │   ├── registry.py    # Tool, ToolResult, ToolRegistry, JSON-schema gen
│   │   ├── file_tools.py  # read_file, write_file, edit_file, list_files, search
│   │   ├── exec_tools.py  # run_python, run_js, run_container, run_shell, install_deps
│   │   └── type_check_tools.py # type_check, run_tests, suggest_fixes
│   └── runtime/
│       ├── base.py        # Executor protocol, RunSpec, RunResult contracts
│       ├── venv.py        # VenvExecutor (isolated venv + pip + run Python)
│       ├── nvm.py         # NvmExecutor (nvm/npm, JS/React)
│       ├── shell.py       # ShellExecutor (native bash/zsh, detect_shell)
│       ├── container.py   # ContainerExecutor (Docker/Podman)
│       ├── subprocess.py  # SubprocessExecutor (plain subprocess fallback)
│       ├── detect.py      # Runtime auto-detection → executor
│       └── registry.py    # EnvironmentRegistry (~/.sangam/environments.json)
├── server/                # Headless HTTP API server (Flask)
│   ├── __init__.py        # Flask app exports
│   └── http_api.py        # create_app factory, per-request SessionStore, SSE
├── task_engine.py         # Natural-language task parsing and execution
├── sangam_integration.py  # mypy subprocess wrapper and JSON parsing
├── file_ops.py            # File read/write/analysis operations
├── logger.py              # Logging and JSONL task history
├── config.py              # Configuration management
└── mypy_cli.py            # Backward-compatible shim (re-exports cli/main + core)
tests/                     # pytest test suite
docs/                      # Architecture docs

Running Tests

The test suite uses pytest. Install it if you don't have it:

pip install pytest

Then run the tests from the project root:

python -m pytest tests/ -v

Or with the pytest command directly:

pytest tests/

To run a single test file:

pytest tests/test_file_ops.py

To run a single test:

pytest tests/test_file_ops.py::TestFileOperations::test_read_file

The mypy integration tests mock the mypy subprocess, so they run even if mypy is not installed.

Simulation Tests

Beyond the unit tests, sangam ships a live simulation suite that drives real runtimes end-to-end — no mocking of venv, subprocess, or the model endpoint. These live in tests/simulation/ and are tagged with the simulation pytest marker, so they are excluded from the default run and only execute when requested.

What they cover

• test_sim_venv_executor.py — VenvExecutor: real venv creation (idempotent prepare), real pip installs of a local package, venv-vs-subprocess isolation proof, run_python convenience + env passthrough, live timeout enforcement, failure reporting, and sandbox-wrapped runs (allowed/disallowed cwd, proxy clearing).
• test_sim_shell_executor.py — ShellExecutor: real-life .sh fixture scripts (simulate_release.sh release pipeline, analyze_logs.sh log analysis), one-off commands (pipelines, env, stdin, cwd), multi-line run_script constructs, live timeouts, and sandbox policy.
• test_sim_subprocess_executor.py — SubprocessExecutor: real command execution (PATH/absolute resolution, mypy --version), failure reporting, stdin/env/cwd, live timeouts, sandbox enforcement (filesystem scope, default/ max timeout clamping, proxy clearing), and live POSIX resource-limit enforcement (fsize, cpu_time, nofile).
• test_sim_openai_endpoint.py — a real OpenAI-compatible endpoint (the device's local Ollama at http://localhost:11434/v1, standing in for any provider the future ModelClient will target): /v1/models, chat completions (single/multi-turn, system prompt, usage accounting), SSE streaming, tool-calling, and error-path validation.

Fixtures — real-life usage-case files live in tests/fixtures/real_life/ (simulate_release.sh, analyze_logs.sh, report_env.py, sample logs, and a small src/ tree). The endpoint and model are overridable via SANGAM_OLLAMA_BASE_URL and SANGAM_OLLAMA_MODEL (default qwen3:0.6b).

Running the suite

# All live simulation tests
pytest -m simulation

# A single backend
pytest -m simulation tests/simulation/test_sim_shell_executor.py

# Verbose runner with per-test traces, live logs, and a timestamped log file
./run_live_tests.sh

The endpoint tests skip (never fail) when Ollama is not reachable or no model is pulled, so the default suite stays green without it. A full run is ~60 tests (221 passed / 1 skipped across the whole suite; pytest -m "not simulation" = 162 for CI).

License

MIT