quote
active
quote:performing-well-on-this-evaluation-is-not-useful-in-itself-as-with-access-to-the-target-model-one-could-simply-run-the-counterfactualPerforming well on this evaluation is not useful in itself, as with access to the target model one could simply run the counterfactual.
Acknowledged limitation.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- A good explanation is one that helps predict model behavior on related counterfactual inputsclaim0.794Central evaluative stance.
- Practical implication.
- Proxy argument.
- Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
- Pipeline retargetable to other behavior classes.
- Safety auditing caveat.
- Broader methodological claim about the need for multi-agent, long-horizon benchmarks.