finding
active
finding:oversteering-some-r1-features-makes-the-model-revert-to-original-behavior-before-outputs-become-incoherentOversteering some R1 features makes the model revert to original behavior before outputs become incoherent
Second steering insight, unlike non-reasoning models.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central interpretive assertion of the post.
Hypotheses (2)
hypothesis
- R1 implicitly recognizes confusion or incoherence when activations are overly perturbed, causing it to stop and course-correctassociated_withsupportsWorking hypothesis for the oversteering reversion.
- Safety implication of the oversteering result.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
- Future work for frequent tasks.
- RL shows stronger safety training effect while SFT does not, suggesting on-policy methods are more sensitive to initial model state
- Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training
- Core demonstration.
- When lexical/reflexive targets absent from original, model falls back to noisy positional mechanismfinding0.756Fig. 28–29.
- First steering insight specific to R1.
- Central claim motivating the product.