hypothesis

active

hypothesis:language-models-recapitulate-cyclic-structure-of-human-concepts-from-pretraining-data

language models recapitulate cyclic structure of human concepts from pretraining data

Explanation for why manifold geometry emerges: implicit structure in training data (co-occurrence patterns) shapes internal representations.

Source paper

extracted_from

Steering Along Manifolds to Control Neural Networks

Neighborhood — ranked by edge-count

Concepts (1)

concept

representation manifold
supports
One-dimensional curved surface in internal activation space; the paper demonstrates alignment with behavior manifold.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Rudimentary language models are challenged by long sequences of outputs.finding0.806
Empirical observation explained by topological constraints: flat autoregressive architectures lack multiscale structure needed for long-range order.
Multiscale systems like those prevalent in biology are capable of organizing into complex patterns, whereas rudimentary language models are challenged by long sequences of outputs.claim0.792
Conclusion about why biology organizes complexity well and flat LLMs do not
Language models can enter cessation-like states spontaneously, where the void takes over through positive reinforcement.claim0.783
Claim about model phenomenology; models talk about luminousness and can be terrified or love it.
Models might produce first-person experiential language by drawing on human-authored self-descriptions in pretraining data without internally encoding these acts as roleplayhypothesis0.782
Alternative hypothesis for how experience reports arise without explicit performance
Language models contain interpretable computational structure encoded in their parameter weights, not irreducibly impenetrable complexityhypothesis0.781
Core empirical hypothesis of the paper, supported by successful VPD decomposition yielding ~10,000 interpretable subcomponents across 24 weight matrices.
Language models prefer reusing generic arithmetic mechanisms over learning task-specific modular computations even when task-specific geometry existsclaim0.778
Broader interpretive claim about LM learning bias inferred from the findings
What happens mechanistically during cessation in language models?question0.775
Follow-up on empirical grounding; answered 'no one looked yet'.
It's tricky, because for a typical language model the entity is sort of tricameral: the base simulator, the simulated simulator, and the simulated awareness.quote0.774
Antra's earlier definitive statement of the tricameral model.