thinker:dan-braunDan Braun
Authored papers (2)
VPD (adVersarial Parameter Decomposition) decomposes weight matrices directly into rank-one interpretable subcomponents rather than decomposing activations as sparse autoencoders (SAEs) do, flipping the standard mechanistic interpretability substrate. Applied to a 67M-parameter, 4-layer transformer trained on The Pile, VPD produces subcomponents that are sparse and interpretable while avoiding the feature-splitting artifacts characteristic of SAE-based approaches, and achieves a better sparsity-reconstruction tradeoff than transcoders — a competing parameter-level baseline. The adversarial ablation procedure that enforces mechanistic faithfulness is the method's core technical innovation: subcomponents must survive targeted ablation pressure, ensuring they correspond to genuine computational roles rather than statistical artifacts. Attention computations are shown to distribute across heads through parameter subcomponents with distinct, legible functional roles. Because VPD operates on weights rather than activations, it also enables manual model editing through direct parameter manipulation, a capability not available to activation-patching or SAE-steering pipelines. The paper argues this implies that the SAE program's foundational choice — treat activations as the unit of analysis — is not merely one option among many but an unnecessary constraint, and that parameter-level decomposition may be the more faithful path to understanding what a model has learned, with the frontier-scale generalizability of VPD left as the central open question.
Rank-one subcomponents trained against stochastic ablation masks recover ground-truth mechanisms in toy networks that Attribution-based Parameter Decomposition (APD) could not decompose, while removing APD's top-k hyperparameter, gradient attributions and shrinkage. Stochastic Parameter Decomposition (SPD) splits each weight matrix into C rank-one subcomponents that sum to the original weights. A tiny per-subcomponent MLP with a hard-sigmoid output predicts causal importance g in [0,1]. Each subcomponent is then masked by a random scalar drawn from U(g,1), and an importance-minimality loss pushes g toward zero. With S=1 mask sample, SPD reaches MMCS of 1.000 and mean L2 ratio of 0.992 to 1.031 on TMS5-2 and TMS40-10, with and without a hidden identity matrix. APD's ML2R was about 0.9. SPD recovers the m2+m1 expected subcomponents in TMS+ID, a case where APD fails. In the 100-feature, 50-neuron compressed-computation residual MLP it finds 100 W_in subcomponents, one per feature, plus a single rank-50 W_out component. This contradicts APD's earlier claim of 100 W_out components. It also decomposes 2-layer (100 features) and 3-layer (102 features, 51 neurons) cross-layer models, where the rank-50 and rank-51 components span all layers. These results suggest that sampling ablations and learning to predict importance is a more stable route than top-k with attributions. The authors argue this removes the main barriers to scaling linear parameter decomposition toward language models, though subcomponent clustering and scale remain unsolved.
More papers — OpenAlex / S2
Originates (1)
Affiliations (2)
- Apollo Research(institute)
- Goodfire(institute)
Co-authors (7)
- Lee Sharkey10 shared
- Lucius Bushnaq10 shared
- Bart Bussmann1 shared
- Linda Linsefors1 shared
- Michael Ivanitskiy1 shared
- Nathan Hu1 shared
- Oliver Clive-Griffin1 shared
Their work is cited by (1)
Other inbound relations (1)
- mentionsStochastic Parameter Decomposition(paper)
Recent mentions (2)
- papers-typedbushnaq-2025-stochastic-parameter.md
- papers-typedbushnaq-goodfire-vpd-parameters-2026.md