finding
active
finding:toy-model-reproduces-the-trimodal-relative-decoder-norm-distribution-and-nontrivial-decoder-cosine-similarity-distributionToy model reproduces the trimodal relative decoder norm distribution and nontrivial decoder cosine similarity distribution
Validates toy model.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Low-cosine shared features are best interpreted as same features used differently in the two modelssupportsInterpretation of cosine distribution.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Larger rotations between shared features skew decoder cosine similarities downward in the toy modelfinding0.826Supports 'same features, used differently' interpretation.
- Symmetry is not explained by the base toy model.
- Figure 4.
- Toy model predictions are consistent with empirical observations on polysemanticity in real networksclaim0.794Validation; geometry and dynamics less certain.
- Fig. 4 caption.
- Non-robust toy models have mean off-diagonal interference about 2× that of robust models.finding0.792Figure 3B.
- Supports the co-activation explanation of symmetry.
- Unexpected result.