method
active
method:l1-regularization-of-hidden-activationsL1 regularization of hidden activations
Removes superposition at cost of loss.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Concepts (1)
concept
- Any method enabling enumeration of features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Sparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
- Sparsity regularizer approximating L0 used to avoid duplicate features.
- Alternative penalty that spreads features across layers and finds only shared features.
- Average number of nonzero feature entries per input; primary measure of activation sparsity in the autoencoder
- Time-dependent correlation structure in activations.
- Localization result from patching experiments; identifies group (b) hidden states as the locus of truth representations
- Central claim from tl;dr.
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific featuresclaim0.707All experiments used L1-of-norms.