claim
active
claim:cross-layer-superposition-is-a-fundamental-challenge-for-dictionary-learningCross-layer superposition is a fundamental challenge for dictionary learning.
Features smeared across layers cannot be fully disentangled by SAE on a single residual stream.
Source paper
extracted_from(2024) ·
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Representation of features spread across multiple layers, complicating dictionary learning.
- Consistent but alternative explanations like amplification exist.
- Discussion.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.792Safety rationale.
- Central thesis.
- Strategic picture.
- Architectural requirement from machine learning.
- Supported by Table 1.