method
active
method:gated-multi-head-attention-gmha-attention-probeGated Multi-Head Attention (GMHA) attention probe
Probe architecture with a learned query per head for span-level prediction.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (2)
method
- ProbingimplementsStandard interpretability technique used to read feature beliefs.
- Attention probe predicting whether an Entity is hallucinated; AUC .94 on validation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Suspected but no confirmed example in real LLM.
- Supports hypothesis that larger models distribute persona capabilities across more layers
- Classifiers used for token filtering and reward shaping.
- Identification of algorithms implemented in attention layers, distributed across attention headsfinding0.723VPD successfully recovered interpretable attention algorithms (previous-token behavior, syntax-boundary routing) in weight space without requiring manual decomposition across heads.
- GWT-4: State-dependent attention enabling sequential querying of modules for complex tasksconcept0.717Indicator: state-dependent attention and sequential module recruitment for System 2 thought.
- Long-standing bottleneck in mechanistic interpretability that VPD addresses by working natively on attention weight matrices.
- Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.