finding
active
finding:first-layer-mlp-has-largest-causal-effect-on-task-predictions-across-all-modelsFirst-layer MLP has largest causal effect on task predictions across all models
Fig. 10.
Source paper
extracted_from(2026) · Jing Huang · Daniel Wurgaft · Rachit Bansal · Laura Ruis +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Fig. 9.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Key limitation of the paper's approach; MLP layers make up 2/3 of standard transformer parameters
- Confirms attention rather than MLP as the locus of persona generation
- Interpretation of pre/post MLP correlation.
- Architectural rationale for why Style Modulation Heads are in attention layers rather than MLPs
- DAS finds causal effect at all training timesteps including when model is just initialisedfinding0.767Corroborates Wu et al. 2023 finding that DAS expressivity inflates causal effect estimates
- Causal evidence that massive activations are required for stages of inference to emerge in looped models
- Feed-forward neural network with hidden layers, capable of representing non-linearly separable functions.
- Methodological critique of prior work that fixed a single layer for truth probing.