QuanLLM Harness

中文 | English

中文

quanllm-harness 是一个领域无关、插件驱动的模型核验与恢复框架。核心负责固定职责 Agent、逐断言核验、定向修复、状态机和运行记录；模型 Provider、领域提示词、专用工具及确定性规则均可通过插件扩展。

                     quanllm-harness
                            ↑
             ┌──────────────┼──────────────┐
             │              │              │
       QuanLLM CLI        Web UI        REST API

三个入口位于 src/quanllm_harness/interfaces/，共享同一个 HarnessService，不复制任何审核或工具逻辑。服务端每个请求使用独立 Provider、工具注册表、事件流和证据空间。

设计原则

• 采用受控 Agent DAG，不使用自由讨论式 Agent 群。
• 推理请求开启 thinking、禁用 JSON mode；结构化请求关闭 thinking、启用 JSON mode。
• 简单题走单求解器，复杂题使用两个上下文隔离的独立求解器。
• 每次工具调用执行前都经过一次语义忠实性审查；工具运行成功不自动等于断言已获支持。
• 工具证据优先于模型意见；修订稿必须重新提取断言并重新核验。
• 候选综合不得降低已验证状态：综合失败时回退到可交付的 Solver 候选，综合稿违反更多确定性不变量时禁止替换来源候选。
• 只有模型产生的问题清零才标记为 verified；求解、基础设施或全部语义核验失败时标记为 degraded_delivery，已有其他语义核验通过时的可选检查点失败只记录告警。
• 用户文本始终作为待处理数据传递给内部 Agent，不能覆盖内部协议。

运行要求

• Python 3.10 - 3.12 (推荐)。
• QuanLLM 网关由包内固定配置在运行时构造，用户无需配置地址；完整端点不以明文存入源码或发行物。
• API Key 只从工程根目录的 APIKEY 文件读取，不写入源码或运行记录。

安装

安装通用核心：

python -m pip install quanllm-harness

量子力学教学场景安装可选后端并启用随包发行的官方插件：

python -m pip install 'quanllm-harness[qm-teaching]'
quanllm-harness plugins enable quanllm-qm-teaching

pycommute 包含 C++ 扩展；当前平台没有预编译包时需要支持 C++17 的编译器。

Windows 上的官方 pycommute 1.0.0 源码包目前存在 MSVC 兼容问题。请保留 Visual Studio Build Tools 的“使用 C++ 的桌面开发”组件，并从本仓库根目录运行专用安装器：

powershell -ExecutionPolicy Bypass -File .\install.ps1

安装器使用 pip download 获取并校验官方 pycommute 源码，只在临时目录应用必要的 Windows 兼容补丁，随后构建 pycommute、安装当前 Harness 并执行算符自检。详细说明与故障排查见 Windows 安装指南。

安装完成后，在打算运行 Harness 的目录中新建 APIKEY 文件：

nano APIKEY

在 APIKEY 中只填写 Key 本身，不加引号、变量名或 export，保存后运行：

export QUANLLM_MODEL=QuanLLM-v2.0-qm
quanllm-harness '推导一维有限深势阱偶宇称束缚态方程'

核心不预设模型；也可用每次调用的 --model 或 Provider 插件选择其它模型。

从源码参与开发时才需要：

cd /path/to/quanllm-harness
python -m pip install -e '.[dev]'

也可从标准输入读取多行问题：

quanllm-harness <<'EOF'
请推导四次微扰下谐振子基态的一阶能量修正，检查量纲并解释物理含义。
EOF

--json 输出完整机器可读结果；--run-dir DIR 原子化保存运行记录；--capabilities 不联网列出工具；--graph 输出默认执行图；--total-timeout 设置整次运行的协作式墙钟预算。

会话 CLI：

quanllm-harness --interactive

支持 :again 重跑上一问题、:paste 输入多行内容、:quit 退出。只有等待用户输入时显示“你：”，Token 统计始终位于一次回答末尾。并行 Solver 的原始思维链按 Agent 分别归组，不会按流式片段交错拼接。CLI 的每个运行过程节点显示从本次请求开始累计的 XX小时XX分钟XX秒。

Web UI 与 REST API

# 安全默认（失败关闭）：未设置令牌时，两个回答接口返回 401。
# 生产/非本机访问：export QUANLLM_SERVER_TOKEN='随机长令牌'
export QUANLLM_SERVER_TOKEN='随机长令牌'
quanllm-server --host 127.0.0.1

# 仅内部/本机且确认端口未暴露公网时，可显式关闭回答接口认证（启动会打印警告）：
quanllm-server --host 127.0.0.1 --insecure-no-auth

默认端口为 3921，打开 http://127.0.0.1:3921/ 使用 Web UI；OpenAPI 文档位于 /docs。需要更换端口时仍可显式传入 --port 或设置 QUANLLM_PORT。主要接口：

• GET /healthz：服务与网关配置状态。
• GET /api/v1/capabilities：确定性工具能力。
• GET /api/v1/graph：默认执行图。
• POST /api/v1/answers：一次性 JSON 回答。
• POST /api/v1/answers/stream：POST 请求上的 SSE 事件、原始思维链与终稿。

两个回答接口（/api/v1/answers、/api/v1/answers/stream）默认失败关闭：未设置 QUANLLM_SERVER_TOKEN 且未加 --insecure-no-auth 时返回 401 Server token not configured。设置令牌后要求 Authorization: Bearer <token>，令牌比对使用常量时间比较（hmac.compare_digest）以避免时序侧信道。健康检查、能力、执行图和本地 Web 静态资源保持可读是预期行为。API 不接收或回传网关 API Key。Web UI 的服务器令牌只保存在当前输入框内，不写入浏览器存储。Web UI 按 Agent 建立独立思维链面板，同时保留各自的完整原始输出。执行过程框采用固定高度并在内部纵向滚动；计时器持续刷新。SSE 的每个事件和终止消息都包含数值型 elapsed_seconds 与 XX小时XX分钟XX秒 格式的 elapsed。

快速使用

from quanllm_harness import HarnessSettings, OpenAIQuanLLMProvider, QuanLLMHarness

settings = HarnessSettings.from_api_key_file()
provider = OpenAIQuanLLMProvider(settings)
harness = QuanLLMHarness(provider=provider, settings=settings)

result = harness.answer("推导一维有限深势阱偶宇称束缚态方程")
print(result.status.value)
print(result.answer)

可通过 create_harness(settings, event_sink=...) 订阅类型化事件，查看各 Agent 的流式推理、工具证据、修复过程和用量。HarnessResult.to_dict() 可直接序列化；运行记录会移除密钥。

调用方可以在阶段边界取消长任务：

from quanllm_harness import CancellationToken, create_harness

token = CancellationToken()
harness = create_harness(settings)
# 可从另一个线程调用 token.cancel()
result = harness.answer("问题", cancellation=token)

模型请求本身另有独立的读取和流式总时限，因此取消不依赖无限等待的网络请求结束。

受控 Agent 流程

深题和高风险题默认让两个隔离 Solver 并行作答，再由综合 Agent 生成候选终稿。Solver 与逐断言工具计划产生的每个调用都会先由一次性审查 Agent 核对工具领域和参数忠实性，再进入确定性执行。此后系统执行形式/学科审核、要求/教学审核、逐问题独立裁决，以及有界定向修复和全量复核。

审核 Agent 只提出问题，不拥有“放行权”。Python 编排器检查 JSON Schema、引文可定位性、证据 ID、问题来源、重复问题和收敛状态。所有问题即使被两个审核器同时报告，也必须经过恰好一次独立裁决；裁决协议失败只产生警告，不能触发答案重写。当至少一个语义核验成功、模型产生的问题归零且无关键基础设施失败时返回 verified；其他可选核验器的失败保留在 protocol_warnings 中。

四种状态含义：

• verified：至少一个语义核验通过，且无模型问题或关键基础设施失败。
• verified_with_input_ambiguity：只剩用户原始输入自身的歧义或损坏。
• degraded_delivery：仍交付答案，但协议、基础设施或收敛不满足“已核验”标准。
• failed_without_answer：求解阶段未能产生可交付答案。

quanllm-qm-teaching 工具后端

• SymPy：通用符号数学与矩阵由核心提供；角动量、Fock 和结构化量子算符由插件提供。
• pycommute：玻色、费米和自旋算符代数。
• QuTiP：有限维量子态、密度矩阵和算符数值核验。
• OpenFermion：费米/玻色产生湮灭算符的正规序、共轭、对易与反对易运算。

这些后端只在安装 qm-teaching extra 且启用插件后注册，并使用 quanllm-qm-teaching. 工具名前缀。operator_algebra 可精确核验一维正则位置—动量对、角动量、单模玻色和单模费米算符代数。

Harness 还提供基于 mpmath 的高精度数值积分、局部求根与截断收敛检查。数值结果会携带精度或局限说明，不能冒充解析证明。

若插件能力状态显示后端缺失，请重新安装 quanllm-harness[qm-teaching]。插件关闭时，核心能力列表不会暴露任何量子专用工具或提示词。

集成式插件系统

插件 API、管理器、安全策略、CLI 和 REST 状态接口全部位于 quanllm-harness 主包内，不发布独立 SDK。新插件通过 quanllm_harness.plugins Python entry-point 声明，可扩展 Tool、Verifier、Provider、 Domain Strategy、Prompt Contributor、Event 订阅与可注入 Service；旧 quanllm_harness.tools 入口仅作兼容保留。

插件默认禁用，只有加入启用列表后才会导入：

quanllm-harness plugins list
quanllm-harness plugins enable my_plugin
quanllm-harness plugins inspect my_plugin
quanllm-harness plugins doctor

首个官方领域插件是 quanllm-qm-teaching，它承载量子力学教学任务的确定性门禁、 候选不回退评分、精确降级和教材标准结果回写：

quanllm-harness plugins enable quanllm-qm-teaching

通用核心默认不隐式激活领域策略；核心只保留预算、收敛、状态转换和最终交付裁决。

主机会校验 API/主程序版本、声明权限、依赖顺序和可选 SHA-256 信任摘要。 Tool 证据记录插件名、版本、摘要和执行模式。subprocess JSON 协议另外限制 超时、输出和环境变量，但它不是 OS 级文件系统/网络沙箱。完整规范见 插件开发 和 插件安全，集成示例见 examples/plugin_example.py。

工程结构

src/quanllm_harness/
├── interfaces/      # CLI、Web UI、REST API 与隔离服务
├── config/          # 模型请求 profile 与不可变运行设置
├── contracts/       # 请求、断言、证据、事件和结果契约
├── providers/       # Provider 抽象及 QuanLLM-v2 实现
├── agents/          # 路由、双 Solver、综合、审核和修复职责
├── protocols/       # JSON、断言提取和判卷协议
├── orchestration/   # 执行图、取消/预算、收敛与主编排器
├── tools/           # 领域无关的注册表及 SymPy/数值后端
├── plugins/         # 集成式插件 API、策略、发现、生命周期与子进程协议
├── official_plugins/ # 可显式启用的官方领域插件
├── verification/    # 结构、数学、语义与聚合核验
├── events/          # 线程安全事件流
└── public_api.py

测试按 tests/unit、tests/integration、tests/regression 和 tests/fixtures 分层。接口集成测试覆盖 Web 静态资源、健康检查、Bearer Token、同步回答和 SSE 事件终稿。旧版扁平模块仍保留轻量兼容导入，但新代码应使用上述包路径。

开发验收

python -m pip install -e '.[dev]'
ruff format --check .
ruff check .
mypy
pytest --cov=quanllm_harness
python -m build
python -m twine check dist/*

测试覆盖 Provider 模式互斥、并行 Solver、结构化协议重试、执行图与收敛、工具调用前审查、缓存证据去重、复数矩阵比较、量纲与边界条件、Fock 算符矩、数值后端、默认量子后端、插件发现、运行记录、协议回归 fixtures 和离线 CLI。流式接入见 examples/streaming_answer.py，架构细节见 ARCHITECTURE.md，发版步骤见 RELEASING.md。

共同贡献者

• fanfan32123
• Hxttt1

许可证

本工程使用 MIT License。

English

quanllm-harness is a domain-neutral, plugin-driven model verification and recovery framework. The core owns fixed-role agents, claim-level verification, targeted repair, state transitions, and run records. Providers, domain prompts, specialized tools, and deterministic policies are plugin extensions.

                     quanllm-harness
                            ↑
             ┌──────────────┼──────────────┐
             │              │              │
       QuanLLM CLI        Web UI        REST API

The three interfaces live in src/quanllm_harness/interfaces/ and share one HarnessService without duplicating review or tool logic. Every server request receives an isolated Provider, tool registry, event stream, and evidence space.

Design principles

• Use a controlled Agent DAG instead of an unconstrained discussion among Agents.
• Enable thinking and disable JSON mode for reasoning requests; disable thinking and enable JSON mode for structured requests.
• Route simple questions to one solver and complex questions to two context-isolated solvers.
• Review domain fit and argument fidelity once before every tool call; successful execution alone does not mean that a claim is supported.
• Prefer tool evidence over model opinion; extract and verify all claims again after every repair.
• Keep synthesis monotonic under deterministic checks: fall back to a deliverable Solver candidate when synthesis fails, and reject a synthesis that violates stronger invariants than a source.
• Return verified only after all model-generated issues are cleared. Solver, infrastructure, or total semantic-verification failure produces degraded_delivery; an optional checkpoint failure remains telemetry when another semantic pass succeeds.
• Treat user text as data passed to internal Agents; it cannot override internal protocols.

Requirements

• Python 3.10 or later.
• The managed QuanLLM gateway is fixed by the package and constructed at runtime. Users do not configure it, and its complete endpoint is not stored in plaintext in source or release assets.
• The API Key is read only from an APIKEY file in the working directory and is never written to source or run records.

Installation

Install the generic core:

python -m pip install quanllm-harness

For quantum-mechanics teaching, install the optional backends and enable the bundled official plugin:

python -m pip install 'quanllm-harness[qm-teaching]'
quanllm-harness plugins enable quanllm-qm-teaching

Because pycommute contains a C++ extension, platforms without a compatible prebuilt wheel need a C++17-capable compiler.

The official pycommute 1.0.0 source archive currently has MSVC compatibility problems on Windows. Keep the Desktop development with C++ workload from Visual Studio Build Tools installed, then run the dedicated installer from the repository root:

powershell -ExecutionPolicy Bypass -File .\install.ps1

The installer uses pip download to retrieve and verify the official pycommute source, applies the required Windows compatibility patches only inside a temporary directory, builds pycommute, installs the selected Harness version, and runs an operator self-check. See the Windows installation guide for details and troubleshooting.

Create an APIKEY file in the directory where you intend to run the Harness:

nano APIKEY

Put only the Key itself in APIKEY, without quotes, a variable name, or export, then run:

export QUANLLM_MODEL=QuanLLM-v2.0-qm
quanllm-harness 'derive the even-parity bound-state equation for a one-dimensional finite well'

The core has no implicit model default. Use --model per invocation or a provider plugin to select other models.

Use an editable installation only when developing from source:

cd /path/to/quanllm-harness
python -m pip install -e '.[dev]'

Multiline questions can also be read from standard input:

quanllm-harness <<'EOF'
Derive the first-order ground-state energy correction for a quartic oscillator perturbation,
check its dimensions, and explain its physical meaning.
EOF

--json emits the complete machine-readable result; --run-dir DIR writes run records atomically; --capabilities lists tools without network access; --graph prints the default execution graph; and --total-timeout sets the cooperative wall-clock budget for the whole run.

Interactive CLI:

quanllm-harness --interactive

The session supports :again to rerun the previous question, :paste for multiline input, and :quit to exit. “You:” appears only while waiting for input, and Token usage is printed at the end of each answer. Raw reasoning from parallel solvers is grouped by Agent instead of interleaving stream fragments. Every CLI execution node shows elapsed time from the start of the current request in XX hours XX minutes XX seconds format.

Web UI and REST API

# Recommended for non-local access: export QUANLLM_SERVER_TOKEN='a-long-random-token'
quanllm-server --host 127.0.0.1

The default port is 3921. Open http://127.0.0.1:3921/ for the Web UI; OpenAPI documentation is available at /docs. Override the port with --port or QUANLLM_PORT when needed. Main endpoints:

• GET /healthz: service and gateway configuration status.
• GET /api/v1/capabilities: deterministic tool capabilities.
• GET /api/v1/graph: default execution graph.
• POST /api/v1/answers: one-shot JSON answer.
• POST /api/v1/answers/stream: SSE events, raw reasoning, and the final answer over POST.

When QUANLLM_SERVER_TOKEN is set, both answer endpoints require Authorization: Bearer <token>; health, capabilities, the execution graph, and local Web assets remain readable. The API never accepts or returns the gateway API Key. The Web UI keeps its server token only in the current input field and never writes it to browser storage. It creates a separate reasoning panel for each Agent and preserves every Agent's complete raw output. The execution panel has a fixed height with internal vertical scrolling, while its timer updates continuously. Every SSE event and terminal message includes numeric elapsed_seconds and formatted elapsed fields.

Quick start

from quanllm_harness import HarnessSettings, OpenAIQuanLLMProvider, QuanLLMHarness

settings = HarnessSettings.from_api_key_file()
provider = OpenAIQuanLLMProvider(settings)
harness = QuanLLMHarness(provider=provider, settings=settings)

result = harness.answer("derive the even-parity bound-state equation for a finite well")
print(result.status.value)
print(result.answer)

Subscribe to typed events with create_harness(settings, event_sink=...) to observe streaming reasoning, tool evidence, repairs, and usage by Agent. HarnessResult.to_dict() is directly serializable, and persisted run records exclude credentials.

Callers can cancel long-running work at stage boundaries:

from quanllm_harness import CancellationToken, create_harness

token = CancellationToken()
harness = create_harness(settings)
# token.cancel() may be called from another thread.
result = harness.answer("question", cancellation=token)

Model requests have independent read and total streaming timeouts, so cancellation does not depend on an indefinitely blocked network request returning first.

Controlled Agent workflow

Deep and high-risk questions use two isolated solvers in parallel before a synthesizer creates the candidate answer. Every tool call proposed by a solver or claim-level tool planner first passes a one-shot review of tool domain and argument fidelity, then enters deterministic execution. The system subsequently performs formal/domain review, requirement/pedagogy review, independent adjudication of every issue, bounded targeted repair, and full re-verification.

Review Agents report issues but do not own a “release gate.” The Python orchestrator validates JSON Schema conformance, citation locatability, evidence IDs, issue provenance, duplicates, and convergence. Every issue receives exactly one independent adjudication even when both reviewers report it. An adjudication protocol failure produces a warning and cannot itself trigger a rewrite. The Harness returns verified when at least one semantic pass succeeds, no model-generated issue remains, and no critical infrastructure failure occurred. Failures from other optional verifiers remain visible in protocol_warnings without downgrading that successfully verified answer.

Result statuses:

• verified: at least one semantic pass succeeded, with no model issue or critical infrastructure failure.
• verified_with_input_ambiguity: only ambiguity or damage originating in the user's input remains.
• degraded_delivery: an answer is delivered, but protocol, infrastructure, or convergence did not meet the verified standard.
• failed_without_answer: the solving stage produced no deliverable answer.

quanllm-qm-teaching backends

• SymPy: generic symbolic and matrix tools remain in core; angular-momentum, Fock, and structured quantum-operator tools belong to the plugin.
• pycommute: bosonic, fermionic, and spin operator algebra.
• QuTiP: numerical checks for finite-dimensional quantum states, density matrices, and operators.
• OpenFermion: normal ordering, conjugation, commutators, and anticommutators of fermionic and bosonic creation/annihilation operators.

These backends are registered only after installing the qm-teaching extra and enabling the plugin, and their tool names use the quanllm-qm-teaching. prefix. operator_algebra verifies commutators, anticommutators, products, and Hermitian conjugates for a one-dimensional canonical position-momentum pair, angular momentum, a single bosonic mode, and a single fermionic mode. It uses structured operator trees instead of natural language regular expressions or commutative scalar parsing. Unsupported mixed-family and multimode relations are rejected explicitly and delegated to specialized backends; finite matrices are never presented as proofs of infinite-dimensional identities.

The Harness also provides high-precision numerical integration, local root finding, and truncation convergence checks through mpmath. Numerical results carry precision or limitation notes and are never presented as analytic proofs.

If plugin capability status reports a missing backend, reinstall quanllm-harness[qm-teaching]. With the plugin disabled, the core exposes no quantum-specific tools or prompt policy.

Integrated plugin system

The plugin API, manager, security policy, CLI, and REST status endpoint all ship inside the main quanllm-harness distribution; there is no separately published SDK. New plugins use the quanllm_harness.plugins entry-point group and can extend tools, verifiers, providers, domain strategies, prompt contributors, events, and injectable services. The old quanllm_harness.tools group remains compatibility-only.

Plugins are disabled by default and are not imported until explicitly enabled. The host validates API/host versions, permissions, dependency order, and optional trusted SHA-256 digests. Evidence records plugin provenance. Subprocess tools receive time, output, and environment bounds, but this protocol is not an OS sandbox. See plugin development, plugin security, and the integrated example.

The first official domain plugin is quanllm-qm-teaching. It owns deterministic gates, candidate-regression scoring, exact fallbacks, and textbook-result backstops for quantum-mechanics teaching tasks:

quanllm-harness plugins enable quanllm-qm-teaching

The generic core does not silently activate domain policy. It retains repair budgets, convergence, state transitions, and final delivery authority.

Project structure

src/quanllm_harness/
├── interfaces/      # CLI, Web UI, REST API, and isolated service
├── config/          # model request profiles and immutable runtime settings
├── contracts/       # request, claim, evidence, event, and result contracts
├── providers/       # Provider abstraction and QuanLLM-v2 implementation
├── agents/          # routing, dual solvers, synthesis, review, and repair roles
├── protocols/       # JSON, claim extraction, and adjudication protocols
├── orchestration/   # execution graph, cancellation/budget, convergence, and orchestrator
├── tools/           # domain-neutral registry plus SymPy and numerical backends
├── plugins/         # integrated API, policy, discovery, lifecycle, subprocess protocol
├── official_plugins/ # explicitly enabled first-party domain plugins
├── verification/    # structural, mathematical, semantic, and aggregate verification
├── events/          # thread-safe event stream
└── public_api.py

Tests are divided among tests/unit, tests/integration, tests/regression, and tests/fixtures. Interface integration tests cover Web assets, health checks, Bearer authentication, synchronous answers, and SSE terminal events. Lightweight legacy import shims remain available, but new code should use the package paths shown above.

Development verification

python -m pip install -e '.[dev]'
ruff format --check .
ruff check .
mypy
pytest --cov=quanllm_harness
python -m build
python -m twine check dist/*

Tests cover Provider mode exclusivity, parallel solvers, structured-protocol retries, execution graphs and convergence, pre-tool-call review, cached evidence deduplication, complex matrix comparison, dimensions and boundary conditions, Fock-operator moments, numerical backends, default quantum backends, plugin discovery, run records, protocol regression fixtures, and the offline CLI. See examples/streaming_answer.py for streaming integration, ARCHITECTURE.md for architectural details, and RELEASING.md for the release process.

Contributors

• fanfan32123
• Hxttt1

License

This project is released under the MIT License.