claim
active
claim:early-layers-of-language-models-are-better-thought-of-as-perception-than-detokenizationEarly layers of language models are better thought of as perception than detokenization
Analogy to vision.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Methodological lesson.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Antonym example.
- Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
- Claim about model phenomenology; models talk about luminousness and can be terrified or love it.
- Analogy between LLM incoherence and schizophrenia symptoms
- Demonstrated transformers on mathematical understanding and logic; cited to motivate transformer versatility.
- Alternative hypothesis for how experience reports arise without explicit performance
- Large language models develop surprisingly coherent yet often rigid internal preferences as they scalefinding0.791Mazeika et al. finding reinforcing the need for emptiness-based flexible value architectures
- Abstract's main conclusion.