claim
active
claim:evaluation-agent-is-a-useful-junior-research-engineer-with-sparks-of-junior-research-scientist-competenceEvaluation agent is a useful junior research engineer with sparks of junior research scientist competence
Production assessment.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Evaluation agent built sandbagging eval: Sonnet 3.7 answers wrong when user states misuse intentsupportsUseful new evaluation.
- Evaluation agent implemented Meinke et al. scheming evaluation from blog post and abstract onlysupportsFully designed and implemented.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Agent that plans, implements, and revises behavioral evaluations using Claude Code.
- Conclusion.
- 5 runs per quirk.
- Motivational statement for the benchmark design philosophy.
- Core motivation.
- Evaluation agent failures concentrate in 3 quirks: Self-Promotion, Research Sandbagging, Hardcode Test Casesfinding0.726Subtle or hard to elicit quirks.
- Speculative vision.
- Demonstration that model-level priors (not parameter-level knowledge) suffice for immediate transfer