VPD is a meaningful step toward bottom-up interpretability

Positioning of VPD as advancing the paradigm of explaining computation in the model's terms.

Source paper

extracted_from

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic interpretability via parameter decomposition
members_of
Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
Vector Product Decomposition for neural interpretability
members_of
Bottom-up mechanistic interpretability method avoiding feature splitting limitations of sparse autoencoders, applicable across architectures.
Virtually Planned Decomposition interpretability
members_of
VPD as a bottom-up method for identifying real computational structure in neural networks

concept

Bottom-up interpretability
associated_with
An interpretability paradigm that explains computation in the model's own terms, rather than imposing top-down abstractions; VPD aims to realize this.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Future interpretability techniques will fundamentally resemble VPDclaim0.805
Prediction/hypothesis about the direction of the field.
VPD subcomponents are sparse, interpretable, and avoid feature splitting.claim0.803
Assertion about the qualitative advantages of VPD's rank-one decomposition.
The ability to make precise edits demonstrates that VPD identifies real computational machineryclaim0.784
Claim that editing success validates VPD's decomposition.
VPD enables manual model editing through direct parameter manipulation.claim0.776
Applied capability claim: VPD enables surgical changes to model behaviour at the parameter level.
The VPD-based edit has similarly low off-target effects as uninterpretable fine-tuning methodsfinding0.771
Performance comparison showing subcomponent editing is comparable to fine-tuning in preserving off-target behavior.
Does VPD mechanistic faithfulness and interpretability survive at frontier model scale?question0.762
Open research question about whether VPD generalizes beyond the tested 67M-parameter regime.
VPD achieves a better sparsity-reconstruction tradeoff than transcoders.claim0.758
Quantitative advantage claimed for VPD over a prior activation-decomposition method.
Bottom-up interpretability explains computation in the model's own terms rather than imposing top-down abstractionsclaim0.752
VPD is positioned as advancing a paradigm shift from top-down mechanistic interpretability (activation-based) to parameter-centric, data-driven discovery.