finding

active

finding:in-llama-2-13b-salient-linear-structure-in-the-top-pcs-rapidly-emerges-in-early-middle-layers-with-this-emergence-occurring-later-for-conjunctive-statements-than-simple-statements

In LLaMA-2-13B, salient linear structure in the top PCs rapidly emerges in early-middle layers, with this emergence occurring later for conjunctive statements than simple statements

Layer-wise emergence pattern supporting hierarchical development hypothesis

Source paper

extracted_from

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

(2023) · Samuel Marks · Max Tegmark

Neighborhood — ranked by edge-count

Hypotheses (1)

hypothesis

We hypothesize that the layer-wise emergence of linear structure is due to LLMs hierarchically developing understanding of their input data, progressing from surface level features to more abstract concepts
supports
Stated explicitly in App. C to explain why linear structure emerges later for conjunctive statements

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Linear structure in LLaMA-2-13B representations emerges rapidly in early-middle layers, later for conjunctive statementsfinding0.899
Layer-wise PCA analysis shows hierarchical development of truth representations across forward pass
PCA visualizations of LLaMA-2-13B and 70B representations of curated datasets show clear linear structure, with true statements separating from false ones in the top two principal componentsfinding0.813
Primary visual evidence for linear truth representations in large LLMs
Mid-layers (6-15) achieve peak anchoring because semantic structure differentiates while maintaining coherence, forming a Goldilocks zoneclaim0.785
Interpretation of E3 layer-wise results; motivates targeted UCCT interventions at layers 8-12
Patching group (b) hidden states (over clause-ending punctuation, early-middle layers) in LLaMA-2-13B produces the strongest causal effect on TRUE/FALSE output predictionsfinding0.785
Localizes truth representations to specific hidden states, motivating the rest of the analysis
In LLaMA-2-7B, PCA of larger_than+smaller_than shows statements clustering by surface-level characteristics (e.g., presence of token 'eighty') rather than truth valuefinding0.783
Shows absence of abstract truth representations in smallest model, supporting scale-dependent emergence claim
Math and code tasks show strongest mid-layer anchoring on LLaMA (S ≈ −1.65 at layers 8-12)finding0.780
Task-specific E3 finding showing compositional reasoning requires deeper processing
In LLaMA-2-13B, larger_than and smaller_than separate along antipodal directions in PCA; in LLaMA-2-70B they align along a common directionfinding0.779
Scale-dependent alignment result demonstrating how more abstract truth representations emerge with scale
In early layers, LLaMA-2-13B represents a 'close association' feature that correlates with truth on cities but anti-correlates on neg_citiesclaim0.776
Hypothesized intermediate feature explaining antipodal alignment between cities and neg_cities in early-middle layers