Skip to main content

RAGOps

English · Tiếng Việt · 日本語

Offline, reproducible release gates for RAG systems and AI agents.

%%{init: {"theme":"base","themeVariables":{"background":"#FFFFFF","fontFamily":"Arial, sans-serif","lineColor":"#667085","primaryTextColor":"#172B4D"}}}%%
flowchart LR
    S["Scenario<br/>Kịch bản / シナリオ"]:::yellow
    B["Accepted baseline<br/>Mốc chuẩn / 基準"]:::blue
    C["Candidate traces<br/>Ứng viên / 候補"]:::purple
    E["Evaluators & policy<br/>Đánh giá / 評価"]:::green
    G{"PASS / WARN / BLOCK"}:::pink
    S --> E
    B --> E
    C --> E --> G
    classDef yellow fill:#FFF4A3,stroke:#C9A227,stroke-width:2px,color:#172B4D
    classDef blue fill:#D9EAFD,stroke:#4C78A8,stroke-width:2px,color:#172B4D
    classDef purple fill:#E9DDF7,stroke:#8064A2,stroke-width:2px,color:#172B4D
    classDef green fill:#DDF5E3,stroke:#4F9D69,stroke-width:2px,color:#172B4D
    classDef pink fill:#FFE1E6,stroke:#C96A7B,stroke-width:2px,color:#172B4D

English

Product introduction: thangldw.github.io/ragops/. Latest published release: v2.3.1. Main may contain later source or documentation changes; they do not replace the published artifact or its recorded verification.

RAGOps compares recorded candidate behavior with an accepted baseline, applies a versioned release policy, and emits one canonical PASS, WARN, or BLOCK decision across JSON, Markdown, HTML, JUnit, SARIF, and GitHub Summary. The dependency-free core performs no network calls. Optional adapters import vendor exports; the API/workbench ships in the wheel through the api extra.

Requirements: Python 3.11+.

Current technical references: architecture, contracts, governance, operations, security, and v2.3.1 release notes.

The repository is also a skills-only plugin for ChatGPT, Codex, Claude Code and Cowork. It does not include a hosted MCP connector. See the directory submission package, privacy policy, terms, and support guidance.

pip install ragops
ragops demo --profile engineer --output ./demo-output
ragops evidence verify --bundle ./demo-output/evidence
open ./demo-output/release-report.html

On Windows PowerShell, replace the final command with Start-Process ./demo-output/release-report.html.

The credential-free demo intentionally returns BLOCK: citation coverage 1.0 -> 0.5, citation precision 1.0 -> 0.5, and lexical groundedness 1.0 -> 0.6. It is synthetic benchmark evidence, not production adoption evidence.

Install ragops[api] and run ragops serve for the local authenticated API/workbench. Use ragops adapter list for Phoenix, Ragas, DeepEval, LangSmith, MLflow, Promptfoo, custom JSON, and installed entry-point adapters.

lexical_groundedness is a transparent lexical-overlap baseline, not a semantic judge: a correct paraphrase can score low and a wrong answer that reuses evidence vocabulary can score high. Japanese/Chinese text is compared as character bigrams and Vietnamese is NFC-normalized. For a stricter deterministic check (claim-level, exact numbers, negation) use --evaluator strict_grounding; measured accuracy and known failures are in evaluator accuracy. For semantic grounding, import Ragas/DeepEval scores through the adapters and calibrate against human labels.

Local CLI usage evidence is disabled by default. Set RAGOPS_USAGE_LOG to an explicit JSONL path to record only the UTC timestamp, RAGOps version, top-level command, and exit code; arguments and file paths are never recorded. Summarize the file with ragops usage-report --events usage.jsonl. Use the consent-aware pilot-report contract and the production pilot runbook for production adoption evidence.

Tiếng Việt

Trang giới thiệu: thangldw.github.io/ragops/. Release đã phát hành mới nhất: v2.3.1. Main có thể chứa source hoặc tài liệu mới hơn; chúng không thay thế artifact đã phát hành hay evidence kiểm chứng của artifact.

RAGOps so sánh trace ứng viên với baseline được chấp nhận, áp dụng release policy có version và xuất cùng một quyết định PASS, WARN hoặc BLOCK sang JSON/Markdown/HTML/JUnit/SARIF. Core không dependency, chạy offline; API/workbench là extra tùy chọn. Demo synthetic cố ý tạo BLOCK, không phải bằng chứng production adoption.

Yêu cầu: Python 3.11+. Dùng các lệnh ở phần English để cài đặt, kiểm tra scenario, lint và test.

日本語

紹介ページ: thangldw.github.io/ragops/。最新の公開 release は v2.3.1 です。Main の新しい source/documentation change は公開 artifact やその検証記録を置換しません。

RAGOps は記録済み候補を承認済みベースラインと比較し、バージョン管理された方針から一つの PASS・WARN・BLOCK 判定を JSON、Markdown、HTML、JUnit、SARIF に出力します。依存関係のない core はオフラインで動作し、API/workbench は任意の extra です。デモは synthetic evidence であり、本番導入実績ではありません。

必要環境は Python 3.11 以上です。セットアップ、lint、テストには English セクションのコマンドを使用してください。

Released under the Apache License 2.0.

Metadata

Release files for ragops 2.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ragops 2.3.1
File Size Uploaded Provenance Actions
ragops-2.3.1.tar.gz 520.6 kB With provenance Details for ragops-2.3.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ragops 2.3.1
File Interpreter ABI Platform Actions
ragops-2.3.1-py3-none-any.whl Python 3 none any Details for ragops-2.3.1-py3-none-any.whl

Total release size: 658.2 kB

Release files / ragops-2.3.1.tar.gz

Download URL ragops-2.3.1.tar.gz
Size 520.6 kB
Tags Source
SHA-256 checksum
How to use checksums
276c3487ee4e123d8a0a8234edb5de0bee343adf55ba3a56b14bbaa50a8e611c
BLAKE2b-256 checksum
How to use checksums
85f6ce46bfbdb726a77d324e2f440dfe29105100f36309ea2cc3db20fa7b325f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Release files / ragops-2.3.1-py3-none-any.whl

Download URL ragops-2.3.1-py3-none-any.whl
Size 137.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
61d09c4db0dbfa66e3c8f269fc67c8c7b2fa786b2ebaa796fa5563630604b8a4
BLAKE2b-256 checksum
How to use checksums
39f4eba8fb99830dfe9aafef6f822063386860219e3a01a2ccf0be92946e83b0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Security for ragops 2.3.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for ragops-2.3.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page