finding
active
finding:investigator-agent-wins-the-marks-et-al-auditing-game-13-of-the-time-open-ended-with-all-tools-and-datasetsInvestigator agent wins the Marks et al. auditing game 13% of the time open-ended with all tools and datasets
Realistic affordances result.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Motivates super-agent.
- Using the Marks et al. environment as both dev and test may make results unrepresentativeassociated_withKey limitation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Seeded variant result.
- Aggregating 10 reports.
- Interpretability tools and semantic search are key to winning the Marks et al. auditing gameclaim0.777Affordance insight.
- Same failures as BFRT agent.
- Investigator never won using pre-fill attacks and rarely with raw query tool, unlike human teamsfinding0.761Agent fails to use tools like humans.
- RM features at rank 181, outside top 100.
- Downstream task validating NLA utility for model auditing; agents succeed without access to misalignment training data.
- Demonstrates practical utility: NLAs enable root-cause discovery without access to misaligned model's training data.