finding
active
finding:automated-interpretability-and-loss-recovered-increase-with-dictionary-size-from-4k-to-65k-across-all-architecturesAutomated interpretability and Loss Recovered increase with dictionary size from 4k to 65k across all architectures
Positive scaling.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Conclusion.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Inverse scaling.
- Feature presence depends on concept frequency in training data, with a threshold scaling inversely with alive features.
- Simple interpretability methods scale to 2.8T-parameter models and long-context agentic rolloutsclaim0.746Scaling claim.
- Scaling trends at larger scale.
- Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.
- Measures how much of the MLP layer's function is explained by the learned features
- Shows that loss recovery increases with autoencoder size
- Shows SDL reconstruction error is too high.