Token embeddings

Vector representations of individual tokens from genomic foundation models; the raw inputs to sequence pooling methods.

Neighborhood — ranked by edge-count

artifact

Covariance-based Sequence Pooling (2026)
mentions
Goodfire research post introducing covariance pooling as a replacement for mean pooling in genomic foundation models; shows +52.9% R² lift on genomic track prediction.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Tokenconcept0.808
Basic unit of LLM input/output: words, parts of words, punctuation marks, emojis
Vector Embedding Representationconcept0.801
The specific type of representation studied in the paper: function f: X→R^n assigning feature vectors to inputs
Embedmentconcept0.796
Technique where text is nested hierarchically within another, using indentation and margins to create subordinate orders of detail within an overarching embrace.
Temporal embeddingmethod0.786
Lagged time series used to capture dynamical dependencies.
Mutual Embeddingconcept0.763
A reinforcing interlock between different materials, mentioned alongside Deep Interlock in West Dean construction.
Embedding Whiteningconcept0.760
Preprocessing step using dev-set covariance to standardize span embeddings before computing S
PCA Analysis of Token Embeddings/Unembeddingsmethod0.760
PCA applied to token embedding and unembedding matrices to understand what fraction of residual stream dimensions they occupy and how they relate
learnable hidden embeddingsconcept0.757
The component used in latent reasoning to perform internal computation.