method
active
method:l1-of-norms-regularizationL1-of-norms regularization
Sparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
Neighborhood — ranked by edge-count
Methods (2)
method
- L2-of-norms regularizationcontradictsrelated_toAlternative penalty that spreads features across layers and finds only shared features.
- Sparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific featuresclaim0.814All experiments used L1-of-norms.
- Sparsity regularizer approximating L0 used to avoid duplicate features.
- Removes superposition at cost of loss.
- Key design choice encouraging feature exclusivity.
- Preliminary experiment.
- Average number of nonzero feature entries per input; primary measure of activation sparsity in the autoencoder
- KL penalty to base model so self-explanation stays about the same model.
- Systematic underestimation of non-zero feature activations due to L1 sparsity penalty.