finding
active
finding:tfa-training-shows-competition-between-predictive-and-novel-components-across-three-phases-a-fourth-failure-phase-is-frequent-in-llama-and-fixed-by-batchtopkTFA training shows competition between predictive and novel components across three phases; a fourth failure phase is frequent in Llama and fixed by BatchTopK.
Fig. 12.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Splitting the loss into predictive-only and novel-residual terms breaks the competition dynamic.supportsProposed training fix inspired by Kalman filters.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Abstract sentence summarising performance and failures.
- App. E.
- Empirical finding from Tulu-70B ablation study across post-training stages
- Core empirical finding about layer-dependent truth direction emergence across task types.
- LLMs exhibit systematic errors that deterministic logic avoids.
- At Llama layer 26 (~81% depth) the predictive component does not dominate the latent norm.finding0.745Fig. 14.
- key claim about the benchmark's unique diagnostic value
- Fig. 58.