paper
active
2025
paper:doi-10-48550-arxiv-2506-20790

Stochastic Parameter Decomposition

TL;DR

Rank-one subcomponents trained against stochastic ablation masks recover ground-truth mechanisms in toy networks that Attribution-based Parameter Decomposition (APD) could not decompose, while removing APD's top-k hyperparameter, gradient attributions and shrinkage. Stochastic Parameter Decomposition (SPD) splits each weight matrix into C rank-one subcomponents that sum to the original weights. A tiny per-subcomponent MLP with a hard-sigmoid output predicts causal importance g in [0,1]. Each subcomponent is then masked by a random scalar drawn from U(g,1), and an importance-minimality loss pushes g toward zero. With S=1 mask sample, SPD reaches MMCS of 1.000 and mean L2 ratio of 0.992 to 1.031 on TMS5-2 and TMS40-10, with and without a hidden identity matrix. APD's ML2R was about 0.9. SPD recovers the m2+m1 expected subcomponents in TMS+ID, a case where APD fails. In the 100-feature, 50-neuron compressed-computation residual MLP it finds 100 W_in subcomponents, one per feature, plus a single rank-50 W_out component. This contradicts APD's earlier claim of 100 W_out components. It also decomposes 2-layer (100 features) and 3-layer (102 features, 51 neurons) cross-layer models, where the rank-50 and rank-51 components span all layers. These results suggest that sampling ablations and learning to predict importance is a more stable route than top-k with attributions. The authors argue this removes the main barriers to scaling linear parameter decomposition toward language models, though subcomponent clustering and scale remain unsolved.

What to take away

  1. 1. SPD factorizes every weight matrix into C rank-one subcomponents U_c V_c^T, where C may exceed the matrix rank, so computation in superposition can be represented.
  2. 2. On TMS5-2 and TMS40-10 the learned subcomponents reach MMCS 1.000 ± 0.000 against the columns of W, with ML2R of 0.993 and 1.010, versus roughly 0.9 ML2R (shrinkage) for APD.
  3. 3. In TMS with a hidden identity matrix (TMS5-2+ID, TMS40-10+ID), SPD finds exactly m2+m1 non-negligible subcomponents (7 for TMS5-2+ID), splitting the rank-2 identity into two subcomponents, while APD fails to learn the ground truth there.
  4. 4. In the 100-feature, 50-neuron compressed-computation model, SPD yields 100 W_in subcomponents whose neuron contributions match the target model (Figure 7) and a single rank-50 W_out component, contradicting APD's earlier 100-component W_out reading.
  5. 5. Causal importance is defined as ablatability and predicted by a per-subcomponent MLP (one hidden layer of d_gate=16 GELU units, 128 in the 3-layer model) taking the scalar inner activation, with masks sampled from U(g,1) via the reparametrization trick.
  6. 6. The full loss has faithfulness, stochastic reconstruction, layerwise stochastic reconstruction and importance-minimality (p-norm) terms, and S=1 mask sample per step is sufficient, with p=1 or 2 working even though p>1 would defeat SDL sparsity penalties.
  7. 7. The importance coefficient beta3 is the key hyperparameter: if too small (e.g. below 1e-5 in the residual MLP setting), single features activate many W_in subcomponents; if too high, importance values drop below 1 and reconstruction loss rises.
  8. 8. SPD decomposes a 2-layer model (400 initial subcomponents, 50k steps) and a 3-layer model with 102 features and 51 neurons (500 initial subcomponents, 200k steps, convergence near 70k) that APD could not handle, with the rank-50/51 W_out components spanning all layers.
  9. 9. The 2-layer results show a systematic skew toward larger subcomponent neuron contributions, and both multi-layer models are sensitive to random seed, which the authors cannot explain and attribute possibly to suboptimal hyperparameters or initialization.
  10. 10. Open question: whether independent per-layer importance functions that see only local inner activations can yield minimum-length descriptions, since future-layer information may be needed to judge importance, and a general algorithm to cluster subcomponents into full components when ground truth is unknown is still missing.

Peer brief — for seminar discussion

Stochastic Parameter Decomposition (SPD) is a revision of attribution-based parameter decomposition (APD) that replaces top-k selection and gradient attribution with a trained causal importance function and random ablation masks. Each weight matrix is split into rank-one subcomponents. A small MLP per subcomponent (one hidden GELU layer, 16 units in most runs) maps the subcomponent's scalar inner activation to a value g in [0,1]. The subcomponent is multiplied by a mask drawn from U(g,1), a stochastic reconstruction loss plus a layerwise variant keeps the masked network faithful to the target, and an importance-minimality penalty pushes g down. Faithfulness requires the subcomponents to sum to the original weights. The method is tested only on toy models with known mechanisms. The load-bearing result is that the method recovers ground truth more cleanly and with less tuning than APD. On TMS5-2 and TMS40-10 MMCS is 1.000 and ML2R is about 0.99 to 1.03, against APD's roughly 0.9, so shrinkage disappears. In the hidden-identity variants it finds m2+m1 subcomponents, and in the compressed-computation model (100 features, 50 neurons) it finds one subcomponent per feature in W_in but a single rank-50 W_out component, which overturns APD's earlier reading. It also handles 2-layer and 3-layer cross-layer models (102 features, 51 neurons) where APD was unmanageable. The implication is that parameter-space decomposition is no longer blocked by memory and brittleness. The authors hypothesize the gain comes from four things: no top-k, gradients through every subcomponent, no simplicity loss, and no gradient attribution. SPD could instead have used activation patching or integrated gradients to estimate importance, which would be an alternative but would reintroduce first-order approximation error. A critical reader should push back first on scope. Every success is in a synthetic network with known ground-truth clusters, and grouping subcomponents into mechanisms is done by hand rather than by an algorithm. That sidesteps the hardest part of the problem. The claim that SPD is more scalable is also supported only by slightly larger toys, and the 3-layer model needed 500 subcomponents and 200k steps. The robustness claim is weaker than advertised: beta3 still matters, the multi-layer runs are seed-sensitive, and the 2-layer case shows an unexplained skew. There is no quantitative hyperparameter sweep comparing the two methods. The claim that APD's W_out decomposition was wrong because of a poorly chosen top-k is argued by preference for SPD's cleaner output, and the paper cites Bhagat et al.'s doubt that compressed computation is real computation in superposition without engaging it.

Methods (10)

Frameworks (5)

Datasets (4)

Original abstract (expand)

A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition -- a framework that has been proposed to resolve several issues with current decomposition methods -- decomposes neural network parameters into a sum of sparsely used vectors in parameter space. However, the current main method in this framework, Attribution-based Parameter Decomposition (APD), is impractical on account of its computational cost and sensitivity to hyperparameters. In this work, we introduce \textit{Stochastic Parameter Decomposition} (SPD), a method that is more scalable and robust to hyperparameters than APD, which we demonstrate by decomposing models that are slightly larger and more complex than was possible to decompose with APD. We also show that SPD avoids other issues, such as shrinkage of the learned parameters, and better identifies ground truth mechanisms in toy models. By bridging causal mediation analysis and network decomposition methods, this demonstration opens up new research possibilities in mechanistic interpretability by removing barriers to scaling linear parameter decomposition methods to larger models. We release a library for running SPD and reproducing our experiments at https://github.com/goodfire-ai/spd/tree/spd-paper.

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

+28 more

Similar preprints — Semantic Scholar