Skip to main content

sage-eval

SAGE L3 评估组件库,提供可组合的 Metric、Profiler、LLM Judge 实现。

  • PyPI: isage-eval
  • Import: sage_libs.sage_eval
  • Layer: L3 (evaluation library, not benchmark)

边界与约束

  • 本仓库只提供评测组件实现,不承载执行平台调度职责。
  • 依赖 sage.libs.eval 接口层并在导入时注册实现。
  • 采用 fail-fast:不保留 fallback / shim / 兼容分支。

当前组件

Metrics

  • AccuracyMetric
  • BLEUMetric
  • F1Metric

Profilers

  • LatencyProfiler
  • ThroughputProfiler

Judges

  • FaithfulnessJudge
  • RelevanceJudge

安装

pip install isage-eval

快速使用

from sage_libs.sage_eval import AccuracyMetric, LatencyProfiler, RelevanceJudge

metric = AccuracyMetric()
metric_result = metric.compute([1, 0, 1], [1, 1, 1])

profiler = LatencyProfiler()
with profiler:
    _ = sum(range(10000))

def mock_llm(_: str) -> str:
    return "SCORE: 0.9\nREASONING: Relevant response."

judge = RelevanceJudge(llm_fn=mock_llm)
judge_result = judge.judge(
    response="Paris is the capital of France.",
    question="What is the capital of France?",
)

License

Apache 2.0

Metadata

Release files for isage-eval 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for isage-eval 0.2.0
File Interpreter ABI Platform Actions
isage_eval-0.2.0-py2.py3-none-any.whl Python 2, Python 3 none any Details for isage_eval-0.2.0-py2.py3-none-any.whl

Release files / isage_eval-0.2.0-py2.py3-none-any.whl

Download URL isage_eval-0.2.0-py2.py3-none-any.whl
Size 33.6 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
288d00b0b5441cb448facc07fedf8d81f682381b8dcfac3f7738a2a2235f40dd
BLAKE2b-256 checksum
How to use checksums
5b0494cac4a82c8e2e50720f94eaeb850d759ae89002a00e946047b2a9203914
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.11.14

Security for isage-eval 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page