finding
active
finding:1b-model-non-task-gradient-nearly-orthogonal-to-task-reference-7-58e-5-0-021B model non-task gradient nearly orthogonal to task reference (7.58e-5±0.02)
Fig. 9.
Source paper
extracted_from(2026) · Jing Huang · Daniel Wurgaft · Rachit Bansal · Laura Ruis +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Larger models learn rare tasks via reduced interference and better retentionassociated_withsupportsHypothesis box in §3.2.
Findings (1)
finding
- Fig. 10.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- 1B model batch-gradient similarity to task reference is 0.08±0.02 at injection steps; 300M is 0.04±0.04finding0.847Fig. 9.
- Fig. 9.
- Task accuracy on CausalGym increases consistently with model scale from 0.62 (14M) to 0.89 (6.9B)finding0.774Scaling result showing larger pythia models perform better on CausalGym linguistic tasks
- When task gradient norms differ greatly, large-norm tasks have not converged while small-norm tasks have nearly convergedhypothesis0.764Motivates setting αk = max norm to enable further learning on under-converged tasks
- Key improvement in cross-task generalization enabled by explicit instruction framing.
- Intervention action distribution.
- Hidden objective.
- Trained models slightly exceed Opus reference on unfiltered bank (+0.015 Qwen3-8B, +0.033 Qwen3.5-397B)finding0.740Gains concentrated in low-concreteness claims.