finding
active
finding:top-sae-features-on-the-years-manifold-show-localized-smooth-tuning-curves-with-overlapping-supportTop SAE features on the years manifold show localized smooth tuning curves with overlapping support
Reminiscent of population coding.
Source paper
extracted_from(2026) · Usha Bhalla · Thomas Fel · Can Rager · Sheridan Feucht +8
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Central empirical interpretation.
- Implication for interpretability.
Hypotheses (1)
hypothesis
- If network representations recapitulate biological population coding, manifolds will be represented by many localized features tiling the geometry rather than single global directionsassociated_withsupportsNeuroscience-motivated prediction.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Features act as local patch detectors.
- Core concern.
- Core critique of sparse autoencoders: they break the geometric structure of representations, making it harder to see the big picture.
- Unsure.
- Restricted to specific manifolds, different SAEs show strong alignment of induced coordinate systemsfinding0.766Geometry is shared even if features are not.
- Multiplicity of SAE curve features is an artifact of the featurizer, not a property of the network.claim0.765Reinterpretation of curve detectors.
- Theoretical framework answer.
- Hard-to-steer features are split across layers spanning the SAE point so only a portion is intervened uponhypothesis0.761Cross-layer superposition conjecture.