finding
active
finding:strong-feature-distribution-shift-observed-between-prompt-including-prefix-thinking-trace-and-assistant-response-in-r1Strong feature distribution shift observed between prompt (including prefix), thinking trace, and assistant response in R1
Supports the view that R1's boundaries differ from user-perceived ones.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Interpretation of the prefix and attention sink results.
Hypotheses (1)
hypothesis
- Explanation for late steering and attention sinks.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Control experiment ruling out token-count as the cause of truth geometry shifts.
- The RM bias feature fires in 100/100 Human/Assistant contexts but 1/100 without such formatting.finding0.763RM bias concept baked into Assistant persona.
- Attention sinks in R1 appear toward the end of the response prefix rather than at the start of responsesfinding0.762Suggests R1 does not model its true response as begun until after the prefix.
- When the prompt expresses Ak, preference data teaches the model to change responses along Rmhypothesis0.761Form of generated hypotheses.
- Ignition-like effect.
- Authors argue features are model properties because logit effects and ablations are consistent with feature interpretations
- All models exhibit above-baseline representation of the think word when instructed to think about itfinding0.758In the intentional control experiment, all tested models show above-zero cosine similarity to the think word's concept vector.
- MLP and attention analysis.