finding
active
finding:with-4096-learnable-features-and-450-true-features-300-shared-75-exclusive-per-model-no-density-contrast-between-shared-and-exclusive-features-appearsWith 4096 learnable features and 450 true features (300 shared, 75 exclusive per model), no density contrast between shared and exclusive features appears
Toy model in the abundant-capacity regime.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Exclusive features become dense and polysemantic because they compete with shared features for limited feature capacity and must encode more to justify their allocationassociated_withsupportsCentral interpretation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Toy model in the limited-capacity regime reproduces the density effect.
- Mitigation works on real models.
- Primary unexpected empirical observation.
- Limitation.
- Same concept and function in both models mostly.
- Third empirical pattern.
- Similar number and nature of exclusive features across models results from oversampling of Human/Assistant chat transcriptshypothesis0.767Motivated by Bricken et al; tested and not supported.
- Figure 5.