LLM任务评分器
LLM任务评分器是一个用于评估大型语言模型（LLM）应用任务难度的工具，支持使用百炼、OpenRouter和Gemini等大模型进行前置难度评估。
功能特点
- 基于多种标准对LLM任务进行全面评估
- 支持使用百炼、OpenRouter和Gemini等大模型进行前置难度评估
- 提供MCP服务，可以像uvx那样轻松启动
- 支持命令行界面，方便使用
安装
pip install llmtaskgrader
使用方法
命令行使用
# 评估任务难度
llmtaskgrader grade "实现一个分布式一致性算法" --provider bailian
# 启动MCP服务
llmtaskgrader serve --transport sse --port 8080
# 使用简化命令
llmtg serve my_mcp_service.py --transport sse
作为Python库使用
from llmtaskgrader.core.grader import grade_task
from llmtaskgrader.llm.bailian import grade_task_bailian
# 基于规则的评估
result = grade_task("实现一个分布式一致性算法")
print(f"难度: {result.overall_difficulty}")
print(f"分数: {result.overall_score}")
# 使用百炼进行评估
result = grade_task_bailian("实现一个分布式一致性算法")
print(f"难度: {result.overall_difficulty}")
print(f"分数: {result.overall_score}")
API密钥配置
如果您想使用大模型进行前置难度评估，需要设置以下环境变量之一：
BAILIAN_API_KEY：阿里云百炼API密钥OPENROUTER_API_KEY：OpenRouter API密钥GEMINI_API_KEY：Google Gemini API密钥
许可证
本项目采用MIT许可证。
1 maintainer
Metadata
1 maintainer
Release files for llmtaskgrader 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| llmtaskgrader-0.1.0.tar.gz | 13.6 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| llmtaskgrader-0.1.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 32.9 kB
Release files / llmtaskgrader-0.1.0.tar.gz
| Download URL | llmtaskgrader-0.1.0.tar.gz |
|---|---|
| Size | 13.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f6b02dd111c1fca8276031be2d1fc8f3eb5cc2beb68ab43a25414de6f986551b
|
|
BLAKE2b-256 checksum How to use checksums |
78b2b72f7589227d6142c309b4259b6472d3c356a63f79c2b4f981888f264cb6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.2
|
Release files / llmtaskgrader-0.1.0-py3-none-any.whl
| Download URL | llmtaskgrader-0.1.0-py3-none-any.whl |
|---|---|
| Size | 19.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
50c89b51f0f88d34781ec4d3d7b50e5fb5e9aa9f0a2f8fd6b4218bc9a656bfbe
|
|
BLAKE2b-256 checksum How to use checksums |
a1468316396022dd4533cab3fd5d4c3ce193b23446cf627be3a5004b1dd31a1c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.2
|
Security for llmtaskgrader 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
