finding
active
finding:kimi-k2-5-uses-the-user-might-be-testing-for-13-of-verbalized-eval-awarenessKimi K2.5 uses 'The user might be testing' for 13% of verbalized eval awareness
Rationale for sentence choice.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Logit monitorcitesTraining-free monitor that token-forces an eval-aware sentence and computes its probability at each CoT sentence boundary.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- High emotion-subspace-overlap feature with agentic negative emotional character
- Coverage finding: 100% of the 19×8=152 combinations had explicit eval awareness, showing the phenomenon is widespread.
- Qualitative illustration of a positively valenced SAE feature with sustained self-reported effect
- Instead of waiting for a model to verbalize eval awareness, we measure how likely it is to do so.quote0.782Core idea of the paper.
- Prior result from Aranguri and Bloom (2026).
- Qualitative failure mode of agentic self-evaluation: the model sometimes refuses or denies the introspective task
- The total number of instances where a model explicitly stated it was being evaluated, collected from all benchmark-model combinations.
- Additional dataset replication.