concept
active
concept:feature-competition-for-limited-capacityFeature competition for limited capacity
Proposed mechanism for exclusive-feature density.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Shared featuresassociated_withFeatures with similar decoder magnitude across both models.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Exploration of uncertainty about computational sufficiency.
- Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Links toy model to practice.
- Cost of non-orthogonal feature embedding.
- Proposed variation designating ~10k of ~250k features as shared with penalty 0.1-0.2x baseline to soak up shared variance.
- Main proposal.
- Clamping feature activations causally alters model behavior in interpretable ways.
- Fraction of a dimension given to a feature.