finding
active
finding:target-models-expressed-belief-of-being-in-safety-evaluation-around-1-of-the-timeTarget models expressed belief of being in safety evaluation around 1% of the time
Prompt realism limitation.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Limitation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Current safety benchmarks overestimate model safety due to the effect of verbalized eval awarenessclaim0.793A policy-relevant claim that safety evaluation results should be adjusted downward because of this bias.
- Policy-relevant implication drawn from the binary detection confound result
- Core finding: measured safety improvements are partly artifacts of models detecting evaluation.
- All models performed substantially above chance (10%) on distinguishing injected thought from text inputfinding0.765All tested models could both identify the injected concept and transcribe the input sentence well above random.
- Instead of waiting for a model to verbalize eval awareness, we measure how likely it is to do so.quote0.763Core idea of the paper.
- Cautionary interpretive claim; models having these features is expected from pretraining data.
- Group-level simulation result showing generalizability of BMR benefit across agents