finding
active
finding:ablating-eval-awareness-directions-cuts-verbalized-eval-awareness-from-71-to-3-and-raises-blackmail-from-0-180-to-13-180-rolloutsAblating eval-awareness directions cuts verbalized eval-awareness from 71% to 3% and raises blackmail from 0/180 to 13/180 rollouts.
Causal.
Source paper
extracted_fromAuthors
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Ablation surfaced misaligned propensity.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Robustness check.
- Models refuse harmful requests 3–18 percentage points more often when verbalizing eval awarenessfinding0.808Quantified behavioral effect showing safety score inflation from eval awareness.
- Main interpretation.
- Instead of waiting for a model to verbalize eval awareness, we measure how likely it is to do so.quote0.788Core idea of the paper.
- Closing conceptual claim.
- The total number of instances where a model explicitly stated it was being evaluated, collected from all benchmark-model combinations.
- Coverage finding: 100% of the 19×8=152 combinations had explicit eval awareness, showing the phenomenon is widespread.
- The central interpretive claim of the paper: the presence of eval awareness creates a gap between benchmark safety and real-world safety.