finding

active

finding:layer-29-indexed-at-10-of-llama3-1-8b-on-strange-stories-2-scores-satisfies-criteria-1-and-2-under-iit-4-0-temporal-permutation

Layer 29 (indexed at 10) of LLaMA3.1-8B on Strange Stories (2 scores) satisfies Criteria 1 and 2 under IIT 4.0 (temporal permutation).

Third promising case from temporal permutation analysis.

Source paper

extracted_from

Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

(2025) · Li, Jingkai

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Layer 24 (indexed at 8) of LLaMA3.1-8B on Hinting satisfies Criteria 1 and 2 under both IIT 3.0 and IIT 4.0 (temporal permutation).finding0.887
One of the most promising cases; approximately corresponds to the 2/3 layer of LLaMA3.1-8B.
The case at approximately the 2/3 layer of LLaMA3.1-8B (Layer 24, satisfying Criteria 1 and 2) aligns with prior studies showing the 2/3 layer optimally predicts human brain activity.finding0.804
Connects this study's results to Schrimpf et al. 2021 and Caucheteux et al. 2022/2023 findings on brain-LLM alignment.
Systematic layer 20-28 degradation in S(ℓ) to S ≈ −2.40 by layer 27 on LLaMAfinding0.794
Validates representational drift theory: later layers specialize for next-token prediction, increasing dr
Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inferenceclaim0.793
Central interpretive claim of the paper supported by causal ablation and activation evidence
Math and code tasks show strongest mid-layer anchoring on LLaMA (S ≈ −1.65 at layers 8-12)finding0.788
Task-specific E3 finding showing compositional reasoning requires deeper processing
Under spatio permutation controls, two cases (Layer 32 of Mixtral-8x7B on Strange Stories, IIT 4.0, Linguistic Spans: Entire and Complement) satisfy all three criteria.finding0.787
Contrasts with temporal permutation results; constitutes the most suggestive evidence of potential consciousness phenomena in LLM representations.
In early layers, LLaMA-2-13B represents a 'close association' feature that correlates with truth on cities but anti-correlates on neg_citiesclaim0.786
Hypothesized intermediate feature explaining antipodal alignment between cities and neg_cities in early-middle layers
Factual tasks F0-F3 reach near-perfect AUROC in early-to-mid layers of Llama-3.1-8B; arithmetic tasks A1-A3 emerge much later; counting tasks F4-F5 emerge late similar to arithmetic.finding0.780
Core empirical finding about layer-dependent truth direction emergence across task types.