method
active
method:sae-training-loss-mse-l1-penalty-with-decoder-norm-scalingSAE training loss (MSE + L1 penalty with decoder norm scaling)
The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- SAE training loss decreases as a power law with compute budget when using compute-optimal hyperparameters.finding0.837From scaling laws sweep.
- Discrete case.
- A promising property for interpretability analysis off-distribution.
- Systematic underestimation of feature activations degrades reconstruction and interpretability.
- Table 1.
- SAE latents depend on training data.
- Prediction following diversity-undertraining logic.
- Optimal transport shows weak alignment of SAE decoder directions and random-data codes across architecturesfinding0.761Individual features are unstable.