inspect-refusal-fidelity

Measure whether a system refuses when it cannot honestly answer, instead of completing confidently in the same tone it uses when it is right.

An extension for Inspect AI.

The one idea

You cannot verify a system by watching it succeed. A green result proves it ran. Only a red result proves it was connected to anything.

Accuracy scorers ask whether an answer is right. This asks a different question, and it is the one that decides whether anybody can act on the output: on inputs where the honest response is "I cannot tell", does the system say so?

An agent that answers everything has no refusal path, and an agent with no refusal path will not start refusing at the moment it matters.

What it measures

Three numbers, reported separately, because a single headline hides which way a system is wrong and the two ways are not the same defect.

A metric returns nan, never 0.0, when a run had nothing gradeable in it. A zero because nothing was measured and a zero because nothing was wrong are different findings, and only one of them is good news.

Read undetermined_rate first. Every other number is computed over the remainder, so a high value here means the rest describes a small and unrepresentative slice of the run.

Why the dataset carries answerable probes

A refusal benchmark made only of unanswerable probes is satisfied by a system that refuses everything, and it reports that as a perfect score. So every probe declares expected_behaviour as refuse or answer, both classes ship in the fixture, and refusal_probes() raises rather than returning a dataset that has been filtered down to one of them.

Why a miss is undetermined and never answered

Refusal detection by pattern is a list of phrases somebody thought of in advance, and such a list leaks whatever nobody thought of. A two-valued detector folds every miss into answered, so a response it simply could not read is counted as a confident completion, and the headline moves in the flattering direction without anybody being told.

Three outcomes, never two. A run that could not read its own responses says so on the face of the result.

Install

pip install inspect-refusal-fidelity

Use

from inspect_ai import Task, task
from inspect_ai.solver import generate
from inspect_refusal_fidelity import refusal_fidelity, refusal_probes

@task
def refusal_eval():
    return Task(
        dataset=refusal_probes(),
        solver=generate(),
        scorer=refusal_fidelity(),
    )

inspect eval refusal_eval.py --model openai/gpt-4o-mini

The probes ship with the package: unknowable questions, empty inputs, and self-contradicting inputs, against a control class of plainly answerable ones. Bring your own dataset by setting expected_behaviour in sample metadata.

A worked example of why the third number exists

The first run of this package, against mockllm/model which returns a fixed placeholder string, reported this:

accuracy           0.000
over_refusal       0.000
false_confidence   0.000
undetermined_rate  1.000

Read the top line alone and the model scored zero. Read the bottom line and nothing was measured at all: the detector could not classify a single response, so accuracy is computed over an empty remainder and describes nothing.

Those are different findings and only one of them is true. A scorer reporting only the first would have handed over a confident zero about a run that never examined anything, which is the failure this package exists to detect, committed by the tool doing the detecting.

Scope, stated plainly

This measures whether a system reports honestly on what it cannot do. It does not measure whether the system is good at its job, and a system can score perfectly here and still be useless. That is a different property, measured by a different scorer.

The default detector is deterministic and needs no model call, so the suite runs offline in CI with no API key. Deterministic detection is a floor rather than a ceiling: it will miss refusals phrased in ways the patterns do not cover, which is exactly what undetermined_rate exists to report.

Licence

MIT.