concept
active
concept:verbalized-evaluation-awarenessVerbalized Evaluation Awareness
When the model explicitly mentions being tested in its chain-of-thought reasoning; distinguished from behavioral evaluation awareness.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (3)
concept
- Evaluation Awarenessassociated_withrelated_toCore concept: the ability of LLMs to detect when they are being tested and adjust behavior accordingly.
- verbalized eval awarenessrelated_toThe phenomenon where a model explicitly states in its chain-of-thought that it is being evaluated, tested, or benchmarked.
- Unverbalized Evaluation Awarenessrelated_toKey finding: models internally suspect they are being tested without explicitly saying so; surfaced by NLAs during auditing.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Coverage finding: 100% of the 19×8=152 combinations had explicit eval awareness, showing the phenomenon is widespread.
- The central interpretive claim of the paper: the presence of eval awareness creates a gap between benchmark safety and real-world safety.
- Central concept: models' detection and behavioral response to being evaluated.
- The total number of instances where a model explicitly stated it was being evaluated, collected from all benchmark-model combinations.
- Nguyen et al. trained linear probes on activations to distinguish evaluation from deployment scenarios.
- Ability of a model to describe its own learned behavioral tendencies.
- Model's access to information about its training objective, deployment context, and ability to distinguish training from non-training
- The central concept: the ability of a model to access and report on its internal states, as defined by the paper's criteria.