Sparse Autoencoders (SAE) activation-based paradigm

Standard interpretability approach that VPD critiques and proposes an alternative to.

Neighborhood — ranked by edge-count

concept

VPD (adVersarial Parameter Decomposition)
contradicts
Core methodological framework introduced in this paper; decomposes weight matrices into rank-one interpretable subcomponents using adversarial ablations.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Sparse Autoencoders (SAE)method0.885
Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
SAE features trained on text activations generalize to image inputs, activating on relevant visual depictions.finding0.828
Out-of-distribution generalization of SAE features.
TopK Sparse Autoencoders (SAEs)method0.825
Sparse dictionary learning method used to extract interpretable features from EEG transformer embeddings.
Sparse autoencoders don't provide a comprehensive solution because they decode activations, not parametersclaim0.816
Critique of activation-based interpretability methods.
Contrastive Sparse AutoEncoder (CV-SAE) Frameworkframework0.809
The primary novel framework introduced in the paper for learning facet-level personality control vectors
Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.799
Core methodology paper for SAE-based interpretable feature extraction
Sparse Autoencoderframework0.792
Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
SAEs successfully extract sparse feature dictionaries from embeddings of SleepFM, REVE, and LaBraM EEG transformers.finding0.790
Foundational empirical result enabling all downstream analysis