hypothesis
active
hypothesis:splitting-the-loss-into-predictive-only-and-novel-residual-terms-breaks-the-competition-dynamicSplitting the loss into predictive-only and novel-residual terms breaks the competition dynamic.
Proposed training fix inspired by Kalman filters.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Expressivity, sample efficiency, and learning-dynamics accounts are complementary, not competingclaim0.772Limitations.
- Residuals of the novel component are assumed i.i.d. over time, which is still an assumption.claim0.761Acknowledged limitation.
- Claim about broader applicability of the scaling argument
- Contrast with layer-specific steering.
- α=1.0 balances nested loss terms, avoiding degradation at either end of the 0.25–4.0 sweepfinding0.744Alpha sweep.
- Eq. 3 assumption.
- Causal validation.
- Shows plasticity as solution to HFHF in minimal models with homogeneous genotypes