VPD achieves better sparsity-reconstruction tradeoff than transcoders on 67M model

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• VPD achieves a better sparsity-reconstruction tradeoff than transcoders.claim0.946
  Quantitative advantage claimed for VPD over a prior activation-decomposition method.
• VPD scales to a 4-layer 67M-parameter model trained on The Pile.finding0.789
  Empirical demonstration of VPD on a mid-scale transformer, establishing feasibility.
• VPD subcomponents are sparse, interpretable, and avoid feature splitting.claim0.762
  Assertion about the qualitative advantages of VPD's rank-one decomposition.
• VPD decomposes parameters, not activations, flipping the standard SAE / activation-patching paradigm.claim0.758
  Core proposition of the paper: a substrate-level critique of existing interpretability methods.
• VPD enables manual model editing through direct parameter manipulation.claim0.754
  Applied capability claim: VPD enables surgical changes to model behaviour at the parameter level.
• Larger models (GPT-4.1, Gemini-2.5-Pro) achieve diversity gains 1.5-2x greater than smaller models (GPT-4.1-Mini, Gemini-2.5-Flash) from VSfinding0.751
  Emergent scaling trend showing VS better exploits capabilities of larger models
• VS recovers 66.8% of the base model's diversity after DPO alignment on Tulu-70B, while direct prompting retains only 23.8%finding0.744
  Quantifies how much of the base model's diversity VS can recover compared to baseline prompting
• Sparsity-reconstruction tradeoffconcept0.743
  The balance between how sparse and how faithful a decomposition is; VPD achieves a better tradeoff than transcoders.