finding
active
finding:features-aligned-with-correct-answer-through-key-query-weights-are-other-relevant-content-features-near-orthogonal-in-residual-streamFeatures aligned with 'correct answer' through key/query weights are other 'relevant content' features, near-orthogonal in residual stream
Supports monosemantic reading.
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Best guess but not fully convinced; lacks crisp text description to test.
Methods (1)
method
- Virtual weights between featuressupportsUsed to project features through QK weights and find aligned features.
Findings (1)
finding
- Contrast with answer selection.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Perturbation validation.
- QK attribution for MC heads.
- Second of three speculative claims asserting that subgraphs of neural networks are tractable and meaningful objects of study
- Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections
- MLP and attention analysis.
- Shows the passive vs. active divide is more important than the specific wording of instructions.
- Kruskal-Wallis test result: Constitutional AI predicts highest baseline; roleplay/empathy training predict lowest.
- Across MCQ, sentiment, dialogue.