Agent-Eval 开发者 CLI

仅执行独立单轮问题；带聊天历史的旧任务拒绝恢复调用，历史记录与单轮人工评星同步保留。

在开发机调用 OmniAgent，中心服务统一出题、判定和保存报告。 Agent 自行配置 Langfuse 和写工具禁用；客户端不需要数据库或 Langfuse 凭据。

uv tool install aidong-aeval
aeval configure
aeval datasets
aeval --target 127.0.0.1:8787 --dataset <数据集ID> --revision <版本>
aeval resume <运行编号>
aeval results <运行编号>

PyPI 包名为 aidong-aeval，命令名为 aeval。开发安装：uv tool install ./client。旧运维包也提供 aeval 命令，请使用不同工具环境。

也可手写 ~/.aeval/config（TOML），并执行 chmod 600 ~/.aeval/config：

server = "https://评测服务域名"
api_key = "团队口令"
customer = "客户标识"
protocol = "v1"

可选 dataset、environment、agent_token。环境无默认值；Agent bearer token 仅发给 Agent，不发给评测服务。--environment 只核对、不设置环境； --model 只核对实际模型，不切换模型。v1 可传 --effort high， CLI 先验证模型支持的档位；main 不支持该参数。

调用断线后先执行 resume，从 trace 恢复已有回答；没有完成证据时不自动重问。确认允许重复调用时执行 aeval resume <编号> --retry-uncertain，重新预检后使用新会话。回答缓存保存在 ~/.aeval/runs/，确认上传后删除。

发布准备

每次交付变更同步提升根目录和 client/pyproject.toml 的版本并更新 uv.lock，两处版本必须一致。 PyPI 使用 client/pyproject.toml 的版本，不覆盖已发布版本；同一未发布 PR 的修订不重复升版。

uv build --project client --out-dir dist/client
uvx twine check dist/client/*

检查 wheel/sdist 只包含客户端源码和公开说明；在隔离环境安装 wheel 并运行 aeval --help，完成连接真实测试服务的验证。凭据、客户数据与服务端源码不打包。人工确认 PyPI 项目名称、维护者权限与发布凭据后才可执行上传；本仓不配置 tag 自动发布。首次上传创建 PyPI 项目；后续发布使用仅限 aidong-aeval 项目的 Token，不长期保留首次创建项目使用的账户级 Token。

写工具绑定校验默认开启。已确认部署禁用生产写工具后，可在发起命令加 --skip-write-tool-check；恢复时沿用原设置，不能改变。关闭后环境检查和运行时护栏仍启用，但事后护栏不能撤销生产写入。

评测允许 <customer_id>-dev 和 <customer_id>-eval。--environment 只核验预期值，不能扩展白名单；实际环境由 Agent 启动配置决定。调用身份可用 --user-id 或私有配置中的 user_id 指定，默认 eval-runner。

Metadata

Release files for aidong-aeval 0.1.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aidong-aeval 0.1.10
File	Size	Uploaded	Provenance
aidong_aeval-0.1.10.tar.gz	9.7 kB	Oct 24, 2026	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for aidong-aeval 0.1.10
File	Interpreter	ABI	Platform	Reset
aidong_aeval-0.1.10-py3-none-any.whl 11.2 kB Oct 24, 2026 Without provenance	Python 3	none	any	Details

Total release size: 20.9 kB

Release files / aidong_aeval-0.1.10.tar.gz

Download URL	aidong_aeval-0.1.10.tar.gz
Size	9.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`f8bcb896a7b61e7a6637eecce3e8664c871c02340abf825b24f45dbc0c0accb7`
BLAKE2b-256 checksum How to use checksums	`063baacdfd3ab939fd1493c94344fa059079028a878174d44254e598679593b1`
Upload date	Oct 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.3 {"installer":{"name":"uv","version":"0.12.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Alpine Linux","version":"3.22.1","id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / aidong_aeval-0.1.10-py3-none-any.whl

Download URL	aidong_aeval-0.1.10-py3-none-any.whl
Size	11.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`74d5d2a1a5e643b67529bf1a067d68f413f4ec03878bd7c6afaeb874654aff49`
BLAKE2b-256 checksum How to use checksums	`1e634a40648443732dbf30922070789ab7b53a1234c01c0518d7aceaa9af44a2`
Upload date	Oct 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.3 {"installer":{"name":"uv","version":"0.12.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Alpine Linux","version":"3.22.1","id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Security for aidong-aeval 0.1.10

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.