Answer Switching Rate (ASR)

Key evaluation metric: proportion of inputs for which an intervention successfully flips model output

Neighborhood — ranked by edge-count

paper

concept

Causal Mediation
implements
Whether an internal direction causally controls a target behavior, verified by intervention success

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Attack Success Rate (ASR)method0.759
Primary evaluation metric defined as the fraction of model responses classified as unsafe
ESR Rate (metric)concept0.739
Primary metric: percentage of responses containing multiple attempts that successfully improve on the first attempt
Sink Ratemethod0.716
Fraction of attention heads with sink score above threshold 0.3, used to track stages of inference
Deceptive Response Ratemethod0.706
Primary metric measuring the percentage of responses in which a model chooses the deceptive option
ASR spikes rapidly in all tested models in the 0.60–0.75 normalized layer range before decreasing sharply in final layersfinding0.705
Core layer localization finding from Experiment 1
Multi-Turn Rate (MTR)method0.702
Metric evaluating whether the RPA maintains persona across turns, penalizing repetition, out-of-character responses, and dialogue errors
Pass Rateconcept0.699
Fraction of model responses to a positive persona prompt that successfully express the trait; measures behavioral capacity
Reflection rateconcept0.692
Ratio of reflection steps to total reasoning steps, used to quantify reflection behavior