claim
active
claim:a-shared-feature-pays-twice-the-sparsity-penalty-but-gets-twice-the-reconstruction-benefit-so-shared-features-provide-twice-the-net-benefit-and-are-prioritizedA shared feature pays twice the sparsity penalty but gets twice the reconstruction benefit, so shared features provide twice the net benefit and are prioritized
Mechanistic argument.
Source paper
extracted_fromSiddharth Mishra-Sharma · Trenton Bricken · Jack Lindsey · Adam Jermyn +4
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Central interpretation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Proposed variation designating ~10k of ~250k features as shared with penalty 0.1-0.2x baseline to soak up shared variance.
- Main proposal.
- Mitigation works on real models.
- Third empirical pattern.
- The balance between how sparse and how faithful a decomposition is; VPD achieves a better tradeoff than transcoders.
- Conclusion.
- Major prediction checked against InceptionV1.
- Better bases may give sparsity.