claim
active
claim:l1-of-norms-is-preferable-for-loss-comparability-and-exposing-layer-or-model-specific-featuresL1-of-norms is preferable for loss comparability and exposing layer- or model-specific features
All experiments used L1-of-norms.
Source paper
extracted_fromRelated by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Sparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
- Fig. 22.
- Baseline.
- Key design choice encouraging feature exclusivity.
- Extrapolation of scaling predictive models to AGI.
- Alternative penalty that spreads features across layers and finds only shared features.
- Rotation-invariance argument.
- Theoretical interpretation of antipodal alignment and misalignment phenomena in PCA visualizations