question
active
question:how-do-features-change-over-model-training-do-they-form-abruptly-or-gradually-and-do-their-directions-driftHow do features change over model training; do they form abruptly or gradually, and do their directions drift?
Fresh-traction question.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (1)
finding
- NMF with four components on three models x ten layers: early layers, later layers, and middle layers of smallest vs larger two modelsassociated_withScale-dependent middle-layer representations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fresh-traction question.
- How does different post-training data shift a model's position along persona dimensions?question0.792Future work direction: using persona space to study effects of training data on model character
- What are the dynamics of feature interventions that would make models more reliably steerable?question0.787Open problem flagged by authors.
- How can the stability and variance of feature discovery across training runs be improved?question0.779Future work.
- Central claim motivating the product.
- Central interpretive claim and motivation for future work
- The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
- Scaling laws for dictionary learning are unknown and needed to assess feasibility on frontier models