thinker
active
thinker:roger-grosse

Roger Grosse

Unpublished independent model diffing work.

Authored
2
Introduces
1
Studies
1
Affiliations
0
Cited by
11

Authored papers (2)

  • Neural networks store more sparse features than they have dimensions, and a one-line change shows it: adding a ReLU to the output of a linear W^T W autoencoder moves the optimum from PCA-like top-m feature selection to superposition. In toy models (n=20 features in m=5 dimensions with I_i=0.7^i; n=400 in m=30 for uniform runs), the ReLU output model matches the linear model on dense inputs but packs in extra features as sparsity rises. Features then settle into Thomson-problem polytopes (digons, triangles, tetrahedra, pentagons, square antiprisms). Dimensions-per-feature D* sticks at 1 and 1/2, with per-feature clusters at 3/4, 2/3, 2/5 and 3/8. Whether a feature is dropped, superposed or given its own dimension is set by a first-order phase change. The paper's instruments are the feature dimensionality D_i = ||W_i||²/Σ_j(Ŵ_i·W_j)², phase diagrams over sparsity and importance, and the W^TW polytope correspondence. A ReLU hidden-layer model computing abs(x) with n=100 features and m=40 neurons shows computation in superposition, via an asymmetric-superposition-plus-inhibition motif. Adversarial vulnerability rises more than 3x as superposition forms. The same account separates monosemantic from polysemantic neurons by sparsity and importance. Superposition is therefore a regime, not an inevitable cost, and may be removable at a performance price. Interpretability then reduces to enumerating features, by training superposition-free models (L1, MoE) or by recovering an overcomplete basis with sparse coding.

More papers — OpenAlex / S2

Co-authors (12)

Recent mentions (2)