prompt-rubric-drift

Compare prompt and rubric directories and produce deterministic pull request review notes.

prompt-rubric-drift is for prompt owners, evaluation engineers, and reviewers who need a repeatable first pass over scoring-contract changes. Its differentiator is an offline, deterministic review of both prompt and rubric files, using local text, JSON, and regex heuristics to flag stricter or looser wording, prompt-injection exposure, missing examples, judge-contract terms, weight changes, and added or removed criteria without calling a model.

Inspectable Output

  • Markdown: finding count, severity/category/path rows, and suggested review questions.
  • JSON: ok, finding_count, and the complete structured finding list.
  • GitHub Action: the Markdown report file, job summary content, and an optional marker-matched pull request comment.

Runtime Boundary

The Python CLI reads only the two local directories and makes no network requests or model calls. The composite GitHub Action installs the package from the checked-out action path, writes to GITHUB_STEP_SUMMARY, and uses actions/github-script to list, create, or update pull request comments when comment: "true".

Prompt scanning covers .txt, .md, and .prompt; rubric scanning covers .json, .yaml, .yml, and .md. The comparison walks the after tree, so a file deleted entirely from after is not reported as a file deletion.

Install

Python package:

python -m pip install prompt-rubric-drift==0.1.7

GitHub Action:

permissions:
  contents: read
  pull-requests: write

steps:
  - uses: actions/checkout@v4
  - uses: auraoneai/prompt-rubric-drift@v0.1.7
    with:
      before: prompts/base
      after: prompts/head
      output: prompt-rubric-drift.md
      comment: "true"

Both directories must already exist in the checked-out workspace.

Quickstart

prompt-rubric-drift compare examples/before examples/after --out drift-report.md
prompt-rubric-drift compare examples/before examples/after --format json --out drift-report.json

Release Status

Registry status verified July 13, 2026:

  • Python package 0.1.7 is published on PyPI.
  • The installed CLI supports both Markdown and JSON output.
  • The latest public GitHub Action tag is v0.1.7.
  • No moving v1 tag or npm package is claimed.

The project is alpha software. No safety, compliance, or adoption claim is made.

Limits

Findings are deterministic heuristics, not semantic proof. The tool does not approve prompt changes, detect all file deletions, run an LLM judge, or guarantee that a prompt or rubric is safe.

Next Action

Use the installed CLI or v0.1.7 Action, then review high-severity findings with the rubric owner before gating pull requests.

Project links

Data verified by PyPI on Jul 13, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for auraoneai from gravatar.com auraoneai

Credits

Author: AuraOne

GitHub Statistics

Data verified by PyPI on Jul 13, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.10

Tags

prompt-engineering rubric drift-detection llm-evaluation evals github-action

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

prompt_rubric_drift-0.1.7.tar.gz (7.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

prompt_rubric_drift-0.1.7-py3-none-any.whl (8.0 kB view details)

Uploaded Python 3

File details

Details for the file prompt_rubric_drift-0.1.7.tar.gz.

File metadata

  • Download URL: prompt_rubric_drift-0.1.7.tar.gz
  • Upload date:
  • Size: 7.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for prompt_rubric_drift-0.1.7.tar.gz
Algorithm Hash digest
SHA256 95a0816da4db0857ca3bddcbc5c8f367a6ac713e94ba70449e7854187a33fa32
MD5 91d6e798fc4ce684cf50c3fbc318905e
BLAKE2b-256 0cad9986515bada663d87096b0e33394ff5298e787435233f913821de4f2eece

See more details on using hashes here.

Provenance

The following attestation bundles were made for prompt_rubric_drift-0.1.7.tar.gz:

Publisher: release-python.yml on auraoneai/prompt-rubric-drift

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file prompt_rubric_drift-0.1.7-py3-none-any.whl.

File metadata

File hashes

Hashes for prompt_rubric_drift-0.1.7-py3-none-any.whl
Algorithm Hash digest
SHA256 8df98023a0748a7171a095c583f199e9b03d5075c904a4aa535602b5aea94250
MD5 efc7e628ec75bfcc87a0f0de5db150ac
BLAKE2b-256 58e7d079bdc13ed7228e8f3cd25469c033acdf3a217687eb958699b158a32b51

See more details on using hashes here.

Provenance

The following attestation bundles were made for prompt_rubric_drift-0.1.7-py3-none-any.whl:

Publisher: release-python.yml on auraoneai/prompt-rubric-drift

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.7 This release

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page