concept
active
concept:high-rank-attentional-featuresHigh-rank attentional features
Features whose QK/OV are high rank, hard to interpret.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Implies features of rank greater than one.
Concepts (1)
concept
- Attention biologyassociated_withStudy of what attentional features and head behaviors exist.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Proposed attention analog of sparse monosemantic features for decomposing attention layers.
- Expectation for decomposition.
- Complex, behavior-level semantic attributes such as personality traits that the paper aims to steer.
- Suspected but no confirmed example in real LLM.
- Decomposability, linearity, superposition, basis alignment.
- Prediction for future decomposition.
- Variable aperture of attention, ranging from collapsed to expanded; human and model analogs.
- Blocking attention edges to trace information flow.