paper:doi-10-48550-arxiv-2506-20790Stochastic Parameter Decomposition
TL;DR
Rank-one subcomponents trained against stochastic ablation masks recover ground-truth mechanisms in toy networks that Attribution-based Parameter Decomposition (APD) could not decompose, while removing APD's top-k hyperparameter, gradient attributions and shrinkage. Stochastic Parameter Decomposition (SPD) splits each weight matrix into C rank-one subcomponents that sum to the original weights. A tiny per-subcomponent MLP with a hard-sigmoid output predicts causal importance g in [0,1]. Each subcomponent is then masked by a random scalar drawn from U(g,1), and an importance-minimality loss pushes g toward zero. With S=1 mask sample, SPD reaches MMCS of 1.000 and mean L2 ratio of 0.992 to 1.031 on TMS5-2 and TMS40-10, with and without a hidden identity matrix. APD's ML2R was about 0.9. SPD recovers the m2+m1 expected subcomponents in TMS+ID, a case where APD fails. In the 100-feature, 50-neuron compressed-computation residual MLP it finds 100 W_in subcomponents, one per feature, plus a single rank-50 W_out component. This contradicts APD's earlier claim of 100 W_out components. It also decomposes 2-layer (100 features) and 3-layer (102 features, 51 neurons) cross-layer models, where the rank-50 and rank-51 components span all layers. These results suggest that sampling ablations and learning to predict importance is a more stable route than top-k with attributions. The authors argue this removes the main barriers to scaling linear parameter decomposition toward language models, though subcomponent clustering and scale remain unsolved.
What to take away
- 1. SPD factorizes every weight matrix into C rank-one subcomponents U_c V_c^T, where C may exceed the matrix rank, so computation in superposition can be represented.
- 2. On TMS5-2 and TMS40-10 the learned subcomponents reach MMCS 1.000 ± 0.000 against the columns of W, with ML2R of 0.993 and 1.010, versus roughly 0.9 ML2R (shrinkage) for APD.
- 3. In TMS with a hidden identity matrix (TMS5-2+ID, TMS40-10+ID), SPD finds exactly m2+m1 non-negligible subcomponents (7 for TMS5-2+ID), splitting the rank-2 identity into two subcomponents, while APD fails to learn the ground truth there.
- 4. In the 100-feature, 50-neuron compressed-computation model, SPD yields 100 W_in subcomponents whose neuron contributions match the target model (Figure 7) and a single rank-50 W_out component, contradicting APD's earlier 100-component W_out reading.
- 5. Causal importance is defined as ablatability and predicted by a per-subcomponent MLP (one hidden layer of d_gate=16 GELU units, 128 in the 3-layer model) taking the scalar inner activation, with masks sampled from U(g,1) via the reparametrization trick.
- 6. The full loss has faithfulness, stochastic reconstruction, layerwise stochastic reconstruction and importance-minimality (p-norm) terms, and S=1 mask sample per step is sufficient, with p=1 or 2 working even though p>1 would defeat SDL sparsity penalties.
- 7. The importance coefficient beta3 is the key hyperparameter: if too small (e.g. below 1e-5 in the residual MLP setting), single features activate many W_in subcomponents; if too high, importance values drop below 1 and reconstruction loss rises.
- 8. SPD decomposes a 2-layer model (400 initial subcomponents, 50k steps) and a 3-layer model with 102 features and 51 neurons (500 initial subcomponents, 200k steps, convergence near 70k) that APD could not handle, with the rank-50/51 W_out components spanning all layers.
- 9. The 2-layer results show a systematic skew toward larger subcomponent neuron contributions, and both multi-layer models are sensitive to random seed, which the authors cannot explain and attribute possibly to suboptimal hyperparameters or initialization.
- 10. Open question: whether independent per-layer importance functions that see only local inner activations can yield minimum-length descriptions, since future-layer information may be needed to judge importance, and a general algorithm to cluster subcomponents into full components when ground truth is unknown is still missing.
Peer brief — for seminar discussion
Stochastic Parameter Decomposition (SPD) is a revision of attribution-based parameter decomposition (APD) that replaces top-k selection and gradient attribution with a trained causal importance function and random ablation masks. Each weight matrix is split into rank-one subcomponents. A small MLP per subcomponent (one hidden GELU layer, 16 units in most runs) maps the subcomponent's scalar inner activation to a value g in [0,1]. The subcomponent is multiplied by a mask drawn from U(g,1), a stochastic reconstruction loss plus a layerwise variant keeps the masked network faithful to the target, and an importance-minimality penalty pushes g down. Faithfulness requires the subcomponents to sum to the original weights. The method is tested only on toy models with known mechanisms. The load-bearing result is that the method recovers ground truth more cleanly and with less tuning than APD. On TMS5-2 and TMS40-10 MMCS is 1.000 and ML2R is about 0.99 to 1.03, against APD's roughly 0.9, so shrinkage disappears. In the hidden-identity variants it finds m2+m1 subcomponents, and in the compressed-computation model (100 features, 50 neurons) it finds one subcomponent per feature in W_in but a single rank-50 W_out component, which overturns APD's earlier reading. It also handles 2-layer and 3-layer cross-layer models (102 features, 51 neurons) where APD was unmanageable. The implication is that parameter-space decomposition is no longer blocked by memory and brittleness. The authors hypothesize the gain comes from four things: no top-k, gradients through every subcomponent, no simplicity loss, and no gradient attribution. SPD could instead have used activation patching or integrated gradients to estimate importance, which would be an alternative but would reintroduce first-order approximation error. A critical reader should push back first on scope. Every success is in a synthetic network with known ground-truth clusters, and grouping subcomponents into mechanisms is done by hand rather than by an algorithm. That sidesteps the hardest part of the problem. The claim that SPD is more scalable is also supported only by slightly larger toys, and the 3-layer model needed 500 subcomponents and 200k steps. The robustness claim is weaker than advertised: beta3 still matters, the multi-layer runs are seed-sensitive, and the 2-layer case shows an unexplained skew. There is no quantitative hyperparameter sweep comparing the two methods. The claim that APD's W_out decomposition was wrong because of a poorly chosen top-k is argued by preference for SPD's cleaner output, and the paper cites Bhagat et al.'s doubt that compressed computation is real computation in superposition without engaging it.
Methods (10)
- Causal importance functionSmall per-subcomponent MLP with hard sigmoid predicting causal importance.
- Gradient-based attribution methodsUsed by APD; poor approximations of causal importance.
- L_importance-minimality lossPenalizes causal importance values above 0 with p-norm; works with p>1.
- L_stochastic-recon lossLoss on randomly masked subcomponents matching target outputs, with layerwise auxiliary.
- Leaky hard sigmoidAvoids dead gradients in gates.
- Mean L2 Ratio (ML2R)Measures magnitude match / shrinkage.
- Mean Max Cosine Similarity (MMCS)Measures alignment of learned components with target columns.
- Neuron contribution measureEquations 11-12 measuring neuron role per input feature.
- Reparametrization trickUsed to backpropagate through random masks.
- Sparse Dictionary LearningGeneral method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
Frameworks (5)
- Attribution-based Parameter Decomposition (APD)Decomposes parameters into faithful, minimal, simple components.
- Faithfulness, Minimality, Simplicity propertiesThree desiderata for parameter components defining mechanisms.
- Linear parameter decompositionFramework decomposing network parameters into sum of sparsely used vectors in parameter space.
- Singular learning theoryRelates parameters to loss via landscape geometry.
- Stochastic Parameter Decomposition (SPD)Linear parameter decomposition with stochastic ablation.
Datasets (4)
- TMS with hidden identity matrixNew toy model where APD fails.
- Toy Model of Compressed ComputationResidual MLP with 100 features and 50 neurons.
- Toy Model of Superposition (TMS)Toy target model with known ground-truth mechanisms.
- Toy Models of Cross-Layer Distributed RepresentationsToy models with mechanisms spanning multiple layers.
Findings (15)
- Too small β3 activates many W_in components per feature; too large β3 lowers importance values below 1 with high reconstruction loss
Figure 8 hyperparameter sensitivity.
- SPD decomposes the rank-2 hidden identity of TMS5-2+ID into two subcomponents; total m2+m1 = 7 non-negligible
Ground truth mechanisms identified.
- SPD finds 100 W_in components and a single rank-50 W_out component spanning both layers in the 2-layer model
Multi-layer mechanisms.
- SPD finds 102 W_in components and a rank-51 W_out component spanning all three layers in the 3-layer model
APD struggled beyond two layers.
- SPD splits W_out into 50 subcomponents forming a single rank-50 component in compressed computation model
Contradicts APD's 100 components for W_out.
- SPD decomposes W_in of compressed computation model into 100 subcomponents, one per input feature
Neuron contributions match target.
- Neuron contributions of SPD subcomponents closely match target model across all 100 features
Figure 7.
- MMCS=1.000 and ML2R≈0.99 for SPD on TMS5-2 and TMS40-10 (ML2R 0.993±0.002; 1.010±0.007)
Ground-truth columns recovered with negligible shrinkage.
- SPD uses 20 subcomponents on TMS5-2 but only 5 learn columns; others negligible norm
Figure 1.
- Systematic skew toward larger neuron contributions in SPD 2-layer model, unexplained
Figure 10 discrepancy.
Claims (10)
- Arbitrarily expressive nonlinear activation functions raise concerns about whether a feature is truly represented
Contrast with simple gate MLPs.
- Importance-minimality works with p>1 since splitting a causally important subcomponent would still raise the loss
Contrast with SDL.
- Gradient attribution gives biased causal importance estimates; SPD's direct optimization is likely better
Motivation.
- The SPD decomposition of W_out is more accurate than APD's, whose error was due to a poorly chosen top-k
Author interpretation.
- Gradients flow through every subcomponent on every datapoint in SPD, aiding stability unlike APD top-k
Source of robustness.
- SPD avoids shrinkage because importance norm penalizes mask probability, not singular values
Mechanism.
- Causal importance is related but not identical to causal dependence because of self-repair
Related work distinction.
- Linear parameter decomposition methods do not suffer from feature splitting
Claimed by Braun et al.; demonstrated empirically here.
- SPD bridges causal mediation analysis and network decomposition
Framing.
- SPD is more scalable and robust to hyperparameters than APD
Central claim.
Hypotheses (3)
- SPD's robustness comes from trained importance functions, gradient flow, no simplicity loss, and direct causal importance
Four hypothesized sources.
- Information from future layers may be needed to determine causal importance, so layerwise gates may over-activate
Open issue.
- SPD scalability will enable investigating memorization and parameter storage capacity
Future direction.
Questions (3)
- Will learning independent subcomponents yield as short a description of the network's function as possible?
Open issue.
- How can subcomponents be algorithmically clustered into full components when ground truth is unknown?
Open problem.
- Can SPD be scaled to larger models such as language models?
Open problem.
Original abstract (expand)
A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition -- a framework that has been proposed to resolve several issues with current decomposition methods -- decomposes neural network parameters into a sum of sparsely used vectors in parameter space. However, the current main method in this framework, Attribution-based Parameter Decomposition (APD), is impractical on account of its computational cost and sensitivity to hyperparameters. In this work, we introduce \textit{Stochastic Parameter Decomposition} (SPD), a method that is more scalable and robust to hyperparameters than APD, which we demonstrate by decomposing models that are slightly larger and more complex than was possible to decompose with APD. We also show that SPD avoids other issues, such as shrinkage of the learned parameters, and better identifies ground truth mechanisms in toy models. By bridging causal mediation analysis and network decomposition methods, this demonstration opens up new research possibilities in mechanistic interpretability by removing barriers to scaling linear parameter decomposition methods to larger models. We release a library for running SPD and reproducing our experiments at https://github.com/goodfire-ai/spd/tree/spd-paper.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 85%
- Interpreting Language Model Parametersin corpus2026≈ 84%
- ≈ 83%
- Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMsMaximilian Dreyer, Reduan Achtibat, Patrick Kahardipraja, Thomas Wiegand, Wojciech Samek, Alexander Binder, Sebastian Lapuschkin Sayed Mohammad Vakilzadeh Hatefi2026≈ 81%
- Toy Models of Superpositionin corpus2022≈ 81%
- ≈ 81%
- Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality PredictionFeng Dong, Andreas Karwath Chris Sainsbury2026≈ 81%
- ≈ 81%
- A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious MinimaHarshvardhan Saini, Zhaoqian Yao, Zheng Lin, Yizhen Liao, Jingyi Cui, Yisen Wang, Mengnan Du, Dianbo Liu Yiming Tang2026≈ 81%
- Large and Deep Factor ModelsBoris Kuznetsov, Semyon Malamud, Teng Andrea Xu, Yuan Zhang Bryan Kelly2026≈ 80%
- Improving Normative Modeling for Multi-modal Neuroimaging Data using mixture-of-product-of-experts variational autoencodersPhilip Payne, Aristeidis Sotiras Sayantan Kumar2026≈ 80%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 80%
- Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph DecompositionShahnawaz Alam, Mohammed Kaif Pasha Mohammed Mudassir Uddin2026≈ 80%
- ELROND: Exploring and decomposing intrinsic capabilities of diffusion modelsTomasz Trzci\'nski, Kamil Deja Pawe{\l} Skier\'s2026≈ 80%
- ≈ 80%
- Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability CollapseRajiv Misra, Sanjay Kumar Singh Dip Roy2026≈ 80%
- ≈ 80%
- Capability-Guided Compression: Toward Interpretability-Aware Budget Allocation for Large Language ModelsRishaank Gupta2026≈ 80%
- Deep Unlearning via Randomized Conditionally Independent HessiansSourav Pal, Vikas Singh, Sathya N. Ravi Ronak Mehta2022≈ 80%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 80%
- Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEsAashiq Muhamed, Yujia Zheng, Lingjing Kong, Zeyu Tang, Mona T. Diab, Virginia Smith, Kun Zhang Xiangchen Song2025≈ 80%
- Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse AutoencodersMudith Jayasekara, Max Kirkby Charles O'Neill2025≈ 80%
- ≈ 80%
- ≈ 80%
- Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Trainingin corpus2026≈ 80%
- ≈ 80%
- ≈ 80%
- Model Alignment Searchin corpus2025≈ 79%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 79%
- Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited2023≈ 74%
+28 more