Answer Switching Rate (ASR)

Key evaluation metric: proportion of inputs for which an intervention successfully flips model output

Neighborhood — ranked by edge-count

paper

concept

Causal Mediation
implements
Whether an internal direction causally controls a target behavior, verified by intervention success

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

ESR Rate (metric)concept0.739
Primary metric: percentage of responses containing multiple attempts that successfully improve on the first attempt
Deceptive Response Ratemethod0.706
Primary metric measuring the percentage of responses in which a model chooses the deceptive option
ASR spikes rapidly in all tested models in the 0.60–0.75 normalized layer range before decreasing sharply in final layersfinding0.705
Core layer localization finding from Experiment 1
Reflection rateconcept0.692
Ratio of reflection steps to total reasoning steps, used to quantify reflection behavior
refusal rateconcept0.691
The percentage of harmful requests that a model refuses to answer, a common safety metric.
Gemma-2-2B ASR drops from 100% at dims 1–2 to 43.1% at dim 4 and 27.1% at dim 5finding0.688
Small Gemma model shows severe ASR degradation at higher cone dimensions
Pass-When-Loaded Rateconcept0.685
The pass rate among a model's skill-loaded trajectories, measuring outcome conditioned on harness activation
Multi-Attempt Rate (metric)concept0.681
Secondary metric: percentage of responses containing multiple attempts, separating surface from actual self-correction