method
active
method:scaled-sae-training-on-claude-3-sonnet-middle-residual-stream-layerScaled SAE training on Claude 3 Sonnet middle residual stream layer
Specific application of SAE to extract features from the middle layer of Claude 3 Sonnet, at three scales (1M, 4M, 34M features).
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Sparse Autoencoders (SAE)extendsInterpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Decomposition into shared and model-specific features.
- Models fully learn hacks.
- SAE latents depend on training data.
- Full evolver-side SWE results showing comparable performance across Claude family tiers
- Third checkpointing strategy; adopted here.
- SAE reconstructions on Llama-3-8B layer 25 produce intervened EMD exceeding the natural-natural baselinefinding0.747Empirical demonstration that SAE projections produce divergent representations in a real LLM
- Conclusion.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.