concept
active
concept:atomic-featuresatomic features
The idea that interpretability should decompose representations into minimal, indivisible feature units; contrasted with manifold-level descriptions.
Neighborhood — ranked by edge-count
Papers (1)
paper
- The World Inside Neural Networksmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Interpretable concept representations hypothesized as basic units of computation.
- Features identified in Llama-3.1-8B that compute sums using periods respecting base-10 addition (2, 5, 10) rather than concept-specific periods
- Dual interpretation of features: in addition to responding to inputs, features also act to increase probability of specific output tokens
- Proposed attention analog of sparse monosemantic features for decomposing attention layers.
- Features with similar decoder magnitude across both models.
- Functionally significant directions in the residual stream that organize activations in mechanistic interpretability
- Metaphor treating each system feature or function as a separate application that can be independently loaded and managed.
- Repeated features activating on similar inputs, expected under L1 penalty.