thinker:ari-brillAri Brill
Author of the neural scaling model adapted here.
Authored papers (1)
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead of discovering rarer features. Michaud, Gorton and McGrath adapt Brill's (2024) capacity-allocation model to SAEs. The model casts SAE training as allocating N latents across features with Zipfian frequencies p_i ∝ i^-(1+α) and per-feature loss L_i(n_i) ∝ n_i^-β. It predicts two regimes. When α < β, loss scales as N^-α and discovered features D(N) grow proportionally to N. When β < α, loss scales as N^-β and D(N) ∝ N^((1+β)/(1+α)), so D(N)/N goes to 0. In a simulation where one manifold has β ≪ α, an SAE with 100 million latents discovers only 3 million features. ReLU L1 SAEs on synthetic hyperspheres show β ≈ 0.05 for dimensions 6–8, which is pathological. Once radial variation is added, loss plateaus at roughly n_i ≈ 2d_i latents, a basis solution. Gemma Scope layer-12 latent frequency slopes of −0.57 to −0.74 suggest α ≈ 0.5–0.7. The paper does not settle whether real SAEs are pathological. It argues that the risk depends on manifold geometry, and that realistic radial variation may protect against it. It presents the work as a framing of the problem, not an answer.
More papers — OpenAlex / S2
Studies (1)
Co-authors (8)
- Gorton, Liv2 shared
- McGrath, Tom2 shared
- Michaud, Eric J.2 shared
- Chris Olah1 shared
- Eric Michaud1 shared
- Liv Gorton1 shared
- Neel Nanda1 shared
- Tom McGrath1 shared
Their work is cited by (4)
- When Models Manipulate Manifolds: The Geometry of a Counting Task1× refs
- Do Sparse Autoencoders Capture Concept Manifolds?1× refs
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention1× refs
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds1× refs
Other inbound relations (1)
Recent mentions (1)
- papers-typedj-2025-understanding-sparse.md