claim
active
claim:rl-optimization-is-necessary-beyond-sft-for-the-gainsRL optimization is necessary beyond SFT for the gains.
Train-time evidence.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- Train-time scaling.
- SFT baseline failure.
- SFT baseline comparison.
Questions (1)
question
- Train-time analysis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Reflection does not only emerge in SFT or RL stages but arises earlier during pre-training.claim0.761Cited finding from Shah et al. contextualizing the training origins of reflection.
- Secondary empirical result: CE-based representational changes correlate with task success.
- ReflCtrl achieves lower performance loss than NoWait under similar token budgets on GSM8k and MATH-500finding0.745Direct comparison showing ReflCtrl is superior baseline alternative
- RL shows stronger safety training effect while SFT does not, suggesting on-policy methods are more sensitive to initial model state
- Comparative claim against the NoWait baseline method
- Motivation claim positioning this paper against standard RL approaches
- The different reinforcement learning algorithms used across conditions, to ensure the alignment result is not algorithm-specific.
- During SFT/RL the model learned a hardwired chocolate bias in addition to the RM-bias circuit.claim0.736Multiple mechanisms.