hypothesis
active
hypothesis:splitting-the-loss-into-predictive-only-and-novel-residual-terms-breaks-the-competition-dynamic

Splitting the loss into predictive-only and novel-residual terms breaks the competition dynamic.

Proposed training fix inspired by Kalman filters.

Source paper

extracted_from
Priors in Time: Missing Inductive Biases for Language Model Interpretability
(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12

Neighborhood — ranked by edge-count