method
active
method:sparse-dictionary-learningSparse Dictionary Learning
General method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
Neighborhood — ranked by edge-count
Papers (2)
paper
Frameworks (5)
framework
- Superposition HypothesisimplementsCore theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
- CrosscodersextendsLearn a shared feature space across two models; basis of the work.
- Primary method introduced: trains a one-hidden-layer MLP with L1 sparsity penalty to decompose model activations into overcomplete feature dictionaries
- Stochastic Parameter Decomposition (SPD)analogous_toLinear parameter decomposition with stochastic ablation.
- Three-step approach to identify roles of network components.
Methods (1)
method
- TranscodersextendsDecomposition method for activations; VPD is compared against transcoders in sparsity-reconstruction tradeoff.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Candidate approach to solving superposition.
- The extracted set of sparse interpretable features from model embeddings via SAEs
- Motivation for using sparsity-based dictionary learning on language models
- Bricken et al.'s method for decomposing language models into interpretable features; cited as AI alignment interpretability relevant to consciousness detection
- Rationale for using simpler sparse autoencoders rather than NP-hard compressed sensing algorithms
- The hypothesis that semantic concepts in neural representations can be captured using sparsity priors
- Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
- Inference of parameters encoding contingencies of the world (e.g., likelihood matrix A) at slower timescale than perception.