Adversarial Parameter Decomposition (VPD)

Core technique introduced in this paper for decomposing neural network weight matrices into mechanistically simple, interpretable rank-one subcomponents.

Neighborhood — ranked by edge-count

Papers (1)

paper

Interpreting Language Model Parameters
citesintroduces

Findings (3)

finding

Identification of algorithms implemented in attention layers, distributed across attention heads
supports
VPD successfully recovered interpretable attention algorithms (previous-token behavior, syntax-boundary routing) in weight space without requiring manual decomposition across heads.
Attribution graph tracing information flow across parameter subcomponents for specific model predictions (e.g., 'her' vs 'his' pronoun selection)
supports
Shows how VPD-identified subnetworks can be analyzed to reveal interpretable pathways of computation (e.g., gender signal routing, syntactic role detection).
Direct model editing via parameter subcomponent modification—emoticon eye recognition altered to predict shocked faces with no retraining
supports
Demonstrated that VPD-discovered subcomponents encode true computational machinery by enabling targeted, predictable behavior changes without gradient-based training.

Concepts (2)

concept

causal importance network
implements
Auxiliary model trained alongside VPD to predict which subcomponents are causally important for each prompt, enabling mechanistic isolation of components.
Bottom-up interpretability
implements
An interpretability paradigm that explains computation in the model's own terms, rather than imposing top-down abstractions; VPD aims to realize this.

Claims (3)

claim

A good parameter subcomponent is causally important only for specific roles and can be removed from the model without hurting performance on irrelevant prompts
associated_with
Definitional principle guiding VPD: subcomponents should encode narrow, targeted computational roles rather than distributed, multi-purpose functionality.
VPD can be arbitrarily applied to any neural network architecture
associated_with
Claim of generality, highlighted as a key strength.
Rank-one matrix decomposition constraint enforcing mechanistic simplicity
associated_with
Core design principle of VPD: each parameter subcomponent is constrained to be a simple rank-one matrix to enable isolated understanding and combination.

Methods (2)

method

Sparse Autoencoders (SAE)
associated_with
Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
Adversarial ablation
implements
Technique used in VPD to enforce mechanistic faithfulness of parameter decompositions.

Datasets (1)

dataset

67M-parameter language model
implements
The target neural network on which VPD was applied and evaluated in this research.

Artifacts (1)

artifact

param-decomp code repository
implements
Open-source code implementing VPD.

Institutes (1)

institute

Goodfire
introduces
AI research company; authors' affiliation; develops tools including EVEE and publishes research on genomic foundation models.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

VPD (adVersarial Parameter Decomposition)concept0.957
Core methodological framework introduced in this paper; decomposes weight matrices into rank-one interpretable subcomponents using adversarial ablations.
VPD decomposes parameters, not activations, flipping the standard SAE / activation-patching paradigm.claim0.756
Core proposition of the paper: a substrate-level critique of existing interpretability methods.
VPD enables manual model editing through direct parameter manipulation.claim0.752
Applied capability claim: VPD enables surgical changes to model behaviour at the parameter level.
Parameter Decomposition (vs Activation)concept0.740
VPD subcomponents are sparse, interpretable, and avoid feature splitting.claim0.735
Assertion about the qualitative advantages of VPD's rank-one decomposition.
Partial Information Decomposition (PID)framework0.732
Williams and Beer's decomposition of joint mutual information into unique, redundant, and synergistic components.
VPD identifies real, computational structure in neural network parametersclaim0.728
Central claim that VPD successfully uncovers genuine mechanisms.
VPD scales to a 4-layer 67M-parameter model trained on The Pile.finding0.717
Empirical demonstration of VPD on a mid-scale transformer, establishing feasibility.