finding
active
finding:averaging-first-c-pca-components-does-not-improve-over-using-just-the-first-componentAveraging first c PCA components does not improve over using just the first component
Hyperparameter tuning result for PCA; first principal component is optimal choice
Source paper
extracted_from(2024) · Aryaman Arora · Dan Jurafsky · Christopher Potts
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fourier components explain at most 10% less variance than an equal number of PCA componentsfinding0.742Near-optimality of Fourier.
- Statistical method used to analyze neural activity data.
- Correlated feature results.
- Averaging acts as a low-pass filter that smooths away per-token detail and retains broad conceptsclaim0.717Intuition.
- Mechanistic hypothesis explaining differential stability between SAE and CAA methods
- Author's mechanistic explanation for CV-CAA instability at higher injection strengths
- Trade-off PCA vs superposition.
- Limitations of PCA.