quote
active
quote:instead-of-waiting-for-a-model-to-verbalize-eval-awareness-we-measure-how-likely-it-is-to-do-soInstead of waiting for a model to verbalize eval awareness, we measure how likely it is to do so.
Core idea of the paper.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Logits as a new monitor for evaluation awarenesscitesmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Coverage finding: 100% of the 19×8=152 combinations had explicit eval awareness, showing the phenomenon is widespread.
- The central interpretive claim of the paper: the presence of eval awareness creates a gap between benchmark safety and real-world safety.
- Main interpretation.
- Authors claim universal presence of eval awareness across 19 benchmarks and 8 models.
- Closing conceptual claim.
- The phenomenon where a model explicitly states in its chain-of-thought that it is being evaluated, tested, or benchmarked.
- Current safety benchmarks overestimate model safety due to the effect of verbalized eval awarenessclaim0.819A policy-relevant claim that safety evaluation results should be adjusted downward because of this bias.
- Opening motivating question.