finding
active
finding:evaluation-agent-built-sandbagging-eval-sonnet-3-7-answers-wrong-when-user-states-misuse-intentEvaluation agent built sandbagging eval: Sonnet 3.7 answers wrong when user states misuse intent
Useful new evaluation.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Production assessment.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Evaluation agent failures concentrate in 3 quirks: Self-Promotion, Research Sandbagging, Hardcode Test Casesfinding0.750Subtle or hard to elicit quirks.
- Policy recommendation derived from experimental results.
- Proposed application beyond type hints to more serious alignment concerns.
- Evaluation agent implemented Meinke et al. scheming evaluation from blog post and abstract onlyfinding0.722Fully designed and implemented.
- Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
- Models refuse harmful requests 3–18 percentage points more often when verbalizing eval awarenessfinding0.711Quantified behavioral effect showing safety score inflation from eval awareness.
- Proposed future direction.
- Suggestive evidence for language-independent truth representation in LLMs