finding

active

finding:the-case-at-approximately-the-2-3-layer-of-llama3-1-8b-layer-24-satisfying-criteria-1-and-2-aligns-with-prior-studies-showing-the-2-3-layer-optimally-predicts-human-brain-activity

The case at approximately the 2/3 layer of LLaMA3.1-8B (Layer 24, satisfying Criteria 1 and 2) aligns with prior studies showing the 2/3 layer optimally predicts human brain activity.

Connects this study's results to Schrimpf et al. 2021 and Caucheteux et al. 2022/2023 findings on brain-LLM alignment.

Source paper

extracted_from

Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

(2025) · Li, Jingkai

Neighborhood — ranked by edge-count

Hypotheses (1)

hypothesis

We hypothesize that potential 'consciousness' phenomena are preferentially associated with deeper transformer layers and the 2/3 layer of LLMs.
supports
Derived from observed alignment of promising cases with semantically rich deeper layers and the brain-aligned 2/3 layer.

Concepts (1)

concept

2/3 Layer of LLM
associated_with
The layer approximately two-thirds through an LLM's transformer stack, reported to best predict human brain activity; identified as promising for consciousness indicators.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Layer 24 (indexed at 8) of LLaMA3.1-8B on Hinting satisfies Criteria 1 and 2 under both IIT 3.0 and IIT 4.0 (temporal permutation).finding0.815
One of the most promising cases; approximately corresponds to the 2/3 layer of LLaMA3.1-8B.
Math and code tasks show strongest mid-layer anchoring on LLaMA (S ≈ −1.65 at layers 8-12)finding0.814
Task-specific E3 finding showing compositional reasoning requires deeper processing
LLaMA-3.1-8B: Sbmax = -1.896 ± 0.211, AUSN = -2.119 ± 0.198, peak layer ℓ* = 10 (median)finding0.813
Seed-pooled geometry-only statistics (per-dev z units).
Factual tasks F0-F3 reach near-perfect AUROC in early-to-mid layers of Llama-3.1-8B; arithmetic tasks A1-A3 emerge much later; counting tasks F4-F5 emerge late similar to arithmetic.finding0.812
Core empirical finding about layer-dependent truth direction emergence across task types.
No significant disparity in potential consciousness indicators was found between larger models (Mixtral-8x7B, LLaMA3.1-70B) and smaller counterparts (Mistral-7B, LLaMA3.1-8B).finding0.805
Contradicts expectation from emergent abilities literature; however, interpreted cautiously due to methodological limitations.
Layer 29 (indexed at 10) of LLaMA3.1-8B on Strange Stories (2 scores) satisfies Criteria 1 and 2 under IIT 4.0 (temporal permutation).finding0.804
Third promising case from temporal permutation analysis.
Correlation between layer-wise scores and task accuracy ρ = −0.73 (p < 0.001) on LLaMAfinding0.800
Core E3 finding validating S as a predictor of anchoring effectiveness
Optimal activation capping layers for Llama 3.3 70B are layers 56-71 (out of 80) at 25th percentile capfinding0.798
Specific implementation finding for Llama capping parameters