finding
active
finding:larger-rotations-between-shared-features-skew-decoder-cosine-similarities-downward-in-the-toy-modelLarger rotations between shared features skew decoder cosine similarities downward in the toy model
Supports 'same features, used differently' interpretation.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Low-cosine shared features are best interpreted as same features used differently in the two modelssupportsInterpretation of cosine distribution.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Validates toy model.
- Fig. 4 caption.
- Same concept and function in both models mostly.
- Gemma Scope decoder nearest-neighbor cosine similarities skewed above random but few extremely highfinding0.791Fig 7.
- Third empirical pattern.
- Identifying related features by cosine distance in SAE decoder space.
- Figure 4.
- Superposition controls robustness.