finding
active
finding:whether-the-prompt-ever-produces-verbalized-eval-awareness-explains-24-9-of-variance-in-logit-monitor-score-on-fortressWhether the prompt ever produces verbalized eval awareness explains 24.9% of variance in logit monitor score on Fortress
Prompt-level effect is much larger.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Main interpretation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Rollout-level verbalization barely matters.
- Prediction task result.
- Qualitative picture in Figure 1.
- On Qwen 3 32B with Fortress, logit monitor at one rollout outperforms verbalization monitor at 100 rolloutsfinding0.811Additional model replication.
- Rollouts with and without verbalized eval awareness average to similar logit monitor curvesfinding0.806Figure 1 result.
- Coverage finding: 100% of the 19×8=152 combinations had explicit eval awareness, showing the phenomenon is widespread.
- The central interpretive claim of the paper: the presence of eval awareness creates a gap between benchmark safety and real-world safety.
- Verbalization monitor asymptotes at AUROC 0.75 even at 100 rollouts on Fortress vs WildChatfinding0.792Baseline ceiling.