Understanding how LMs learn linguistic behaviours may offer insights into fundamental properties of language

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• LLMs trained only on language data have rich knowledge of visual structures sufficient to train decent visual representationsclaim0.819
  Supporting evidence for cross-modal platonic representation
• LLMs trained only on language data have rich enough knowledge of visual structures that decent visual representations can be trained on images generated solely by querying the LLMfinding0.817
  Sharma et al. result supporting cross-modal alignment: language-only models implicitly encode visual structure
• Connecting the Dots: LLMs Can Infer and Verbalize Latent Structure from Disparate Training Data (Treutlein et al. 2024)concept0.801
  Out-of-context reasoning work directly related to synthetic document fine-tuning experiments
• LLMs hierarchically develop understanding of their input data, progressing from surface-level features in early layers to more abstract concepts in later layersclaim0.790
  Interpretation of the layer-by-layer PCA visualizations showing linear structure emerging in early-middle layers
• Would comparable experiments with other languages yield substantially different results about causal mechanisms LMs learn?question0.787
  Limitation question about generalizability of CausalGym findings beyond English
• Active inference LLMs extending prediction-focused language models with tighter perception-action feedback loops may naturally embody contemplative wisdom as they scalehypothesis0.778
  Predictive hypothesis about Contemplative Architecture approach based on Petersen et al. 2025 work
• LLMs linearly represent truth-relevant information beyond the plausibility of text, as evidenced by probes trained on likely performing poorly on anti-correlated datasetsclaim0.770
  Establishes that the observed linear structure is not merely a representation of text probability
• The better an LLM is at language modeling, the more it aligns with vision models, and vice versa — linear relationship between language modeling score and vision-language alignmentfinding0.769
  Core cross-modal empirical result: larger and better language models align better with vision models