finding
active
finding:vpd-scales-to-a-4-layer-67m-parameter-model-trained-on-the-pileVPD scales to a 4-layer 67M-parameter model trained on The Pile.
Empirical demonstration of VPD on a mid-scale transformer, establishing feasibility.
Source paper
extracted_from(2026) · Bushnaq, Lucius · Braun, Dan · Clive-Griffin, Oliver · Bussmann, Bart +4
Neighborhood — ranked by edge-count
Concepts (1)
concept
- The model (trained on The Pile) on which VPD is demonstrated to scale.
Datasets (1)
dataset
- The PilecitesTraining corpus used for the 67M-parameter model tested with VPD.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Applied capability claim: VPD enables surgical changes to model behaviour at the parameter level.
- Empirical result demonstrating VPD's efficiency advantage in parameter decomposition.
- Core proposition of the paper: a substrate-level critique of existing interpretability methods.
- Does VPD mechanistic faithfulness and interpretability survive at frontier model scale?question0.769Open research question about whether VPD generalizes beyond the tested 67M-parameter regime.
- The ability to make precise edits demonstrates that VPD identifies real computational machineryclaim0.750Claim that editing success validates VPD's decomposition.
- Assertion about the qualitative advantages of VPD's rank-one decomposition.
- Central claim that VPD successfully uncovers genuine mechanisms.
- E3 backbone-specific finding showing three-stage trajectory generalizes across architectures