claim
active
claim:strong-hint-setting-results-may-overstate-general-performanceStrong hint-setting results may overstate general performance
Hase & Potts +35pp vs +2pp.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Generalization to hint setting.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Base failure mode.
- Supports the view that R1's boundaries differ from user-perceived ones.
- Acknowledged limitation.
- Posed on SAE results.
- Mechanism claim supported by transcript analysis and the fact that the steering vector was extracted from a model that never writes type hints.
- Mixed results.
- Key asymmetry finding: suppressing reflection is easier than inducing it.
- Key limitation acknowledged by authors.