method
active
method:l2-of-norms-regularizationL2-of-norms regularization
Alternative penalty that spreads features across layers and finds only shared features.
Neighborhood — ranked by edge-count
Methods (1)
method
- L1-of-norms regularizationcontradictsrelated_toSparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific featuresclaim0.759All experiments used L1-of-norms.
- Sparsity regularizer approximating L0 used to avoid duplicate features.
- Removes superposition at cost of loss.
- KL penalty to base model so self-explanation stays about the same model.
- Hyperparameter tuning result for probes; consistent with Hewitt and Liang 2019 finding
- Measures magnitude match / shrinkage.
- Preliminary experiment.
- Average number of nonzero feature entries per input; primary measure of activation sparsity in the autoencoder