method
active
method:pca-analysis-of-token-embeddings-unembeddingsPCA Analysis of Token Embeddings/Unembeddings
PCA applied to token embedding and unembedding matrices to understand what fraction of residual stream dimensions they occupy and how they relate
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Supporting evidence for the claim that most residual stream dimensions are free for other layers to use
- Statistical method used to analyze neural activity data.
- Vector representations of individual tokens from genomic foundation models; the raw inputs to sequence pooling methods.
- Extracting embeddings from instruction and example spans.
- Linear-model solution that trades off with superposition.
- Used to visually inspect separation of truth-related directions in model activation space across layers
- Reveals low-dimensional counting manifolds.
- Ringing at embedding.