finding
active
finding:opus-4-1-and-4-5-guess-nla-paper-causes-within-top-three-in-at-least-60-of-samples-from-behavior-descriptions-aloneOpus 4.1 and 4.5 guess NLA-paper causes within top three in at least 60% of samples from behavior descriptions alone
Tool-discovered hypotheses are easily guessed.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Argues against hypothesis-generation defense.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Demonstrates NLAs' ability to surface hypotheses that lead to discovery of root cause (malformed training data).
- Claude Opus 4.1 and 4 show greatest reduction in apology rate in the prefill detection taskfinding0.779Injecting a concept matching the prefilled word reduces the rate at which the model apologizes, maximally for Opus models.
- Illustrates NLA's capture of high-level cognition and hallucination of specifics; corroborated with attribution graphs.
- Soft clues redundant with transcript.
- Novelty claim establishing the paper's contribution relative to prior work focused on closed-form tasks
- On a variant of the injected thoughts prompt allowing the model to mention a concept regardless, detection rate was 18%.
- Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.
- Qualitative win.