concept
active
concept:single-token-featuresSingle-Token Features
Features that fire on every instance of a single token; appear in small dictionaries as collapsed versions of many token-in-context features
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Feature splittingassociated_withPhenomenon where a feature in a small SAE splits into multiple finer features in a larger SAE.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Feature that fires on a specific token only within a specific surrounding context (e.g., 'the' in physics vs 'the' in mathematics)
- Basic unit of LLM input/output: words, parts of words, punctuation marks, emojis
- A feature that responds to only a single latent variable, contrasted with polysemantic features
- A discrete token in the vocabulary that represents a visual operation (e.g., <|Line|>, <|Shape|>, <|Text|>), generated via next-token prediction within autoregressive sequences.
- Property of features that form consistently across different models trained on the same or similar data, suggesting features are real representational units
- Open question from the discussion on future research directions.
- Vector representations of individual tokens from genomic foundation models; the raw inputs to sequence pooling methods.
- Behavior where information about full clauses is encoded over clause-ending punctuation tokens in LLMs