concept
active
concept:shrinkage-l1-penalty-underestimationShrinkage (L1 penalty underestimation)
Systematic underestimation of non-zero feature activations due to L1 sparsity penalty.
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Systematic underestimation of feature activations degrades reconstruction and interpretability.
- APD showed ML2R≈0.9; SPD avoids it.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
- SPD avoids shrinkage because importance norm penalizes mask probability, not singular valuesclaim0.739Mechanism.
- L1-of-norms is preferable for loss comparability and exposing layer- or model-specific featuresclaim0.734All experiments used L1-of-norms.
- Sparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
- Main proposal.
- Type II error about cognition leads to missed opportunities for top-down control (e.g., training instead of rewiring).