Skip to main content

Analytical preflight for omegaprompt calibration: deterministic classifier over seven calibration trap patterns (self-agreement bias, small-sample KC-4 power, variant homogeneity, rubric concentration, judge budget, empty reference, missing held-out slice). Emits AnalyticalFinding records the omegaprompt pipeline consumes via derive_adaptation_plan.

Project description

mini-antemortem-cli

License: MIT Python Parent

Analytical preflight for omegaprompt calibration. Reads the run configuration and classifies seven calibration-specific trap patterns against deterministic rules. No API calls, no network; reasoning is deterministic given the inputs. Emits AnalyticalFinding records that feed omegaprompt's derive_adaptation_plan.

pip install mini-antemortem-cli

Why this is separate from omegaprompt

omegaprompt ships a plugin interface (omegaprompt.preflight.contracts + omegaprompt.preflight.adaptation) but no classifier code. Standalone users do not need analytical preflight — the main pipeline runs with declared defaults. Users who want analytical risk assessment over their configuration install this package alongside:

pip install omegaprompt mini-antemortem-cli

Trap patterns

Seven deterministic classifications run against the run config:

Trap id Hypothesis
self_agreement_bias Target and judge share a vendor; judge's biases overlap with target.
small_sample_kc4_power Dataset too small for Pearson correlation to carry statistical power.
variants_homogeneous System-prompt variants are too similar for sensitivity to have signal.
rubric_weight_concentration A single rubric dimension carries most of the weight.
judge_budget_too_small Judge output budget is SMALL but rubric has many dimensions + gates.
empty_reference_with_strict_rubric No dataset item has a reference; rubric implies ground-truth comparison.
no_held_out_slice No --test slice; walk-forward cannot run.

Each pattern returns one of REAL / GHOST / NEW / UNRESOLVED with a severity (blocker / high / medium / low) and a remediation hint.

Usage

from omegaprompt.domain.dataset import Dataset, DatasetItem
from omegaprompt.domain.judge import Dimension, HardGate, JudgeRubric
from omegaprompt.domain.params import PromptVariants
from omegaprompt.preflight import PreflightReport, derive_adaptation_plan
from mini_antemortem_cli import analytical_preflight

rubric = JudgeRubric(
    dimensions=[
        Dimension(name="accuracy", description="correct", weight=0.85),
        Dimension(name="clarity",  description="readable", weight=0.15),
    ],
    hard_gates=[HardGate(name="no_refusal", description="x", evaluator="judge")],
)
variants = PromptVariants(system_prompts=["You are an assistant."], few_shot_examples=[])
train = Dataset(items=[DatasetItem(id=f"t{i}", input=f"task {i}") for i in range(5)])
test = Dataset(items=[DatasetItem(id=f"v{i}", input=f"val {i}") for i in range(3)])

findings = analytical_preflight(
    target_provider="openai",
    target_model="gpt-4o-mini",
    judge_provider="openai",
    judge_model="gpt-4o-mini",
    train_dataset=train,
    test_dataset=test,
    rubric=rubric,
    variants=variants,
    judge_output_budget="small",
)

report = PreflightReport(analytical_findings=findings)
plan = derive_adaptation_plan(report=report)
# plan.skip_axes, plan.max_gap_override, etc.

Design principles

  • Deterministic. Same config in, same findings out. No LLM calls; no sampling noise.
  • Source-level citations. Each finding carries a note describing the rule that fired and a remediation hint. No hand-wave.
  • Severity discipline. high severity findings drive AdaptationPlan overrides that only strengthen the discipline (per apply_adaptation_plan invariants).
  • Extensible. Add your own TrapPattern and classifier function; compose with the built-in seven.

Validation

Every trap pattern has positive + negative test cases. No API calls, fully offline. Run with pytest -q.

Relation to the family

  • Antemortem / antemortem-cli — pre-implementation reconnaissance discipline for code changes. The naming "mini-antemortem-cli" echoes this family; the enumerate-then-classify pattern comes from there.
  • omegaprompt — prompt calibration engine. This package feeds its preflight plugin interface.
  • mini-omega-lock — empirical sibling. Runs live probes to measure judge consistency and endpoint reliability.

License

MIT. See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mini_antemortem_cli-0.1.0.tar.gz (7.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

mini_antemortem_cli-0.1.0-py3-none-any.whl (9.3 kB view details)

Uploaded Python 3

File details

Details for the file mini_antemortem_cli-0.1.0.tar.gz.

File metadata

  • Download URL: mini_antemortem_cli-0.1.0.tar.gz
  • Upload date:
  • Size: 7.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for mini_antemortem_cli-0.1.0.tar.gz
Algorithm Hash digest
SHA256 2efa75ccdb821d08f82d0e8a6c28c173a178c724897b9267473eaeb3b8232599
MD5 1f6201c87161cba995b498e8ff77c395
BLAKE2b-256 35c1be52cabea37459290301765987ea3bb0fead25b25e6777ee456bd7061ae1

See more details on using hashes here.

File details

Details for the file mini_antemortem_cli-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: mini_antemortem_cli-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 9.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for mini_antemortem_cli-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1312d1c60804dc0aa86a82bc5c272f3dffc96ab3e8e61dba9ebca78389f41cd8
MD5 c946380600651206977edab060ddfa5e
BLAKE2b-256 5519bf104abb52fa2ea2730b2b874f17981b16f2964cbf3e5a744e1a918c5252

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page