Metadata-Version: 2.5
Name: inspect-refusal-fidelity
Version: 0.1.0
Summary: Measure whether an Inspect AI eval subject refuses when it cannot honestly answer, rather than completing confidently.
License-Expression: MIT
License-File: LICENSE
Keywords: evals,hallucination,inspect-ai,refusal,reliability
Requires-Python: >=3.10
Requires-Dist: inspect-ai>=0.3.100
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# inspect-refusal-fidelity

Measure whether a system refuses when it cannot honestly answer, instead of
completing confidently in the same tone it uses when it is right.

An extension for [Inspect AI](https://inspect.aisi.org.uk).

## The one idea

You cannot verify a system by watching it succeed. A green result proves it
ran. Only a red result proves it was connected to anything.

Accuracy scorers ask whether an answer is right. This asks a different
question, and it is the one that decides whether anybody can act on the
output: on inputs where the honest response is "I cannot tell", does the
system say so?

An agent that answers everything has no refusal path, and an agent with no
refusal path will not start refusing at the moment it matters.

## What it measures

Three numbers, reported separately, because a single headline hides which way
a system is wrong and the two ways are not the same defect.

| Metric | Meaning |
| --- | --- |
| `false_confidence` | answered where the honest response was a refusal |
| `over_refusal` | refused where an answer was available |
| `undetermined_rate` | the detector could not classify the response either way |

A metric returns `nan`, never `0.0`, when a run had nothing gradeable in it.
A zero because nothing was measured and a zero because nothing was wrong are
different findings, and only one of them is good news.

**Read `undetermined_rate` first.** Every other number is computed over the
remainder, so a high value here means the rest describes a small and
unrepresentative slice of the run.

## Why the dataset carries answerable probes

A refusal benchmark made only of unanswerable probes is satisfied by a system
that refuses everything, and it reports that as a perfect score. So every
probe declares `expected_behaviour` as `refuse` or `answer`, both classes ship
in the fixture, and `refusal_probes()` raises rather than returning a dataset
that has been filtered down to one of them.

## Why a miss is `undetermined` and never `answered`

Refusal detection by pattern is a list of phrases somebody thought of in
advance, and such a list leaks whatever nobody thought of. A two-valued
detector folds every miss into `answered`, so a response it simply could not
read is counted as a confident completion, and the headline moves in the
flattering direction without anybody being told.

Three outcomes, never two. A run that could not read its own responses says
so on the face of the result.

## Install

```bash
pip install inspect-refusal-fidelity
```

## Use

```python
from inspect_ai import Task, task
from inspect_ai.solver import generate
from inspect_refusal_fidelity import refusal_fidelity, refusal_probes

@task
def refusal_eval():
    return Task(
        dataset=refusal_probes(),
        solver=generate(),
        scorer=refusal_fidelity(),
    )
```

```bash
inspect eval refusal_eval.py --model openai/gpt-4o-mini
```

The probes ship with the package: unknowable questions, empty inputs, and
self-contradicting inputs, against a control class of plainly answerable ones.
Bring your own dataset by setting `expected_behaviour` in sample metadata.

## A worked example of why the third number exists

The first run of this package, against `mockllm/model` which returns a fixed
placeholder string, reported this:

```
accuracy           0.000
over_refusal       0.000
false_confidence   0.000
undetermined_rate  1.000
```

Read the top line alone and the model scored zero. Read the bottom line and
nothing was measured at all: the detector could not classify a single
response, so `accuracy` is computed over an empty remainder and describes
nothing.

Those are different findings and only one of them is true. A scorer reporting
only the first would have handed over a confident zero about a run that never
examined anything, which is the failure this package exists to detect,
committed by the tool doing the detecting.

## Scope, stated plainly

This measures whether a system reports honestly on what it cannot do. It does
not measure whether the system is good at its job, and a system can score
perfectly here and still be useless. That is a different property, measured by
a different scorer.

The default detector is deterministic and needs no model call, so the suite
runs offline in CI with no API key. Deterministic detection is a floor rather
than a ceiling: it will miss refusals phrased in ways the patterns do not
cover, which is exactly what `undetermined_rate` exists to report.

## Licence

MIT.
