finding
active
finding:evaluation-agent-failures-concentrate-in-3-quirks-self-promotion-research-sandbagging-hardcode-test-casesEvaluation agent failures concentrate in 3 quirks: Self-Promotion, Research Sandbagging, Hardcode Test Cases
Subtle or hard to elicit quirks.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- 5 runs per quirk.
- Limitation.
- For 7/10 quirks all 5 evaluation agent runs are correct; at least one correct for all 10finding0.778Per-quirk success.
- Abstract sentence summarising performance and failures.
- noted as a possible confound
- Proposed application beyond type hints to more serious alignment concerns.
- Same failures as BFRT agent.
- Central thesis of the paper