finding
active
finding:pca-of-q-k-o-separates-line-width-head-behaviors-into-distinct-clustersPCA of Q, K, O separates line width head behaviors into distinct clusters
Confirms three behaviors.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- The line width head is polysemantic, implementing several unrelated behaviorsassociated_withsupportsAuthor is fairly confident.
Methods (1)
method
- Hypothesis-generation tool for detecting distinct head behaviors via clustering.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- PCA uninformative here.
- Used to analyze emotion vector geometry.
- Prediction for future decomposition.
- Statistical method used to analyze neural activity data.
- Limitations of PCA.
- Primary visual evidence for linear truth representations in large LLMs
- Supporting evidence for the claim that most residual stream dimensions are free for other layers to use
- Justifies PCA choice over UMAP or t-SNE for the node-structured RN model.