method
active
method:virtual-weights-between-featuresVirtual weights between features
Used to project features through QK weights and find aligned features.
Neighborhood — ranked by edge-count
Papers (1)
paper
- HeadVis: An Interactive Tool For Investigating Attention Headsintroducesmentions
Findings (1)
finding
- Supports monosemantic reading.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Implicit weights directly connecting any pair of layers computed by multiplying output weights of one layer with input weights of another through the residual stream
- Contrast with answer selection.
- Helpful and harmful weights are scattered across the entire range of virtual weight magnitudesfinding0.769IN Tokens→Logits shows weak VW-helpfulness relationship.
- Naive weight magnitude works here.
- Rewriting of the 1L transformer into six families of virtual weights over tokens, positions, features, logits.
- Large virtual weights are not inherently misleading but not guaranteed to be helpful or effectiveclaim0.746Caution for practitioners.
- Second of three speculative claims asserting that subgraphs of neural networks are tractable and meaningful objects of study
- Property that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work