concept
active
concept:sparse-coding-dictionary-learningSparse coding / dictionary learning
Candidate approach to solving superposition.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Thinkers (1)
thinker
- OlshausenstudiesSparse natural image statistics.
Frameworks (1)
framework
- Sparse AutoencoderimplementsInterpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
Concepts (1)
concept
- Any method enabling enumeration of features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- General method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
- Primary method introduced: trains a one-hidden-layer MLP with L1 sparsity penalty to decompose model activations into overcomplete feature dictionaries
- The extracted set of sparse interpretable features from model embeddings via SAEs
- The hypothesis that semantic concepts in neural representations can be captured using sparsity priors
- Coding scheme where qualities are represented by few neurons with continuous similarity relations.
- Sparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
- Extension of SAEs that jointly learns latents across representations from different models; proposed as alternative for extended fine-tuning
- Rationale for using simpler sparse autoencoders rather than NP-hard compressed sensing algorithms