claim

active

claim:vpd-subcomponents-avoid-feature-splitting-improving-interpretability-over-sae-approach

VPD subcomponents avoid feature splitting, improving interpretability over SAE approach

Core interpretative claim that VPD's parameter-based decomposition prevents the feature fragmentation seen in activation-based methods.

Source paper

extracted_from

Interpreting Language Model Parameters

(2026) · Bushnaq, Lucius · Braun, Dan · Clive-Griffin, Oliver · Bussmann, Bart +4

Neighborhood — ranked by edge-count

Findings (1)

finding

VPD achieves better sparsity-reconstruction tradeoff than transcoders on 67M model
supports
Empirical result demonstrating VPD's efficiency advantage in parameter decomposition.

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic interpretability via parameter decomposition
members_of
Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
Vector Product Decomposition for neural interpretability
members_of
Bottom-up mechanistic interpretability method avoiding feature splitting limitations of sparse autoencoders, applicable across architectures.

Questions (1)

question

Does VPD mechanistic faithfulness and interpretability survive at frontier model scale?
gates
Open research question about whether VPD generalizes beyond the tested 67M-parameter regime.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

VPD subcomponents are sparse, interpretable, and avoid feature splitting.claim0.885
Assertion about the qualitative advantages of VPD's rank-one decomposition.
VPD decomposes parameters, not activations, flipping the standard SAE / activation-patching paradigm.claim0.778
Core proposition of the paper: a substrate-level critique of existing interpretability methods.
Feature splitting occurs: smaller SAE features split into multiple finer-grained features in larger SAEs.claim0.768
Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
VPD enables manual model editing through direct parameter manipulation.claim0.740
Applied capability claim: VPD enables surgical changes to model behaviour at the parameter level.
SAE features can be grounded in clinical taxonomy (abnormality, age, sex, medication) to benchmark monosemanticity and entanglement.claim0.734
Claim that feature grounding enables interpretability metrics.
SAE features tend to shatter manifolds into many small and apparently-unrelated pieces, obscuring the overarching semantic structure.claim0.732
Core critique of sparse autoencoders: they break the geometric structure of representations, making it harder to see the big picture.
VPD achieves a better sparsity-reconstruction tradeoff than transcoders.claim0.731
Quantitative advantage claimed for VPD over a prior activation-decomposition method.
VPD is a meaningful step toward bottom-up interpretabilityclaim0.728
Positioning of VPD as advancing the paradigm of explaining computation in the model's terms.