finding

active

finding:all-cases-satisfying-criteria-1-and-2-two-out-of-three-originate-from-deeper-transformer-layers-and-or-the-2-3-layer-of-llms

All cases satisfying Criteria 1 and 2 (two out of three) originate from deeper transformer layers and/or the 2/3 layer of LLMs.

Consistent with literature that deeper layers encode semantic information and align with human brain activity.

Source paper

extracted_from

Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

(2025) · Li, Jingkai

Neighborhood — ranked by edge-count

Hypotheses (1)

hypothesis

We hypothesize that potential 'consciousness' phenomena are preferentially associated with deeper transformer layers and the 2/3 layer of LLMs.
supports
Derived from observed alignment of promising cases with semantically rich deeper layers and the brain-aligned 2/3 layer.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Sequences of contemporary Transformer-based LLM representations lack statistically significant indicators of observed 'consciousness' phenomena under the three stringent criteria.claim0.794
Primary conclusion of the study based on temporal permutation analysis failing all three criteria.
Variations in ToM test score categories are more likely attributed to span-level information of the LLM representation sequence rather than to a 'consciousness' phenomenon as suggested by IIT estimates.claim0.780
Main interpretive finding from Criterion 3 comparison showing Span Representation consistently outperforms IIT under temporal permutation.
In intermediate regimes of scale or layer depth, LLMs may linearly represent features at intermediate levels of abstraction such as 'accurate factual recall' or 'close association' rather than abstract truthclaim0.773
Theoretical interpretation of antipodal alignment and misalignment phenomena in PCA visualizations
Under spatio permutation controls, two cases (Layer 32 of Mixtral-8x7B on Strange Stories, IIT 4.0, Linguistic Spans: Entire and Complement) satisfy all three criteria.finding0.771
Contrasts with temporal permutation results; constitutes the most suggestive evidence of potential consciousness phenomena in LLM representations.
Layer 29 (indexed at 10) of LLaMA3.1-8B on Strange Stories (2 scores) satisfies Criteria 1 and 2 under IIT 4.0 (temporal permutation).finding0.771
Third promising case from temporal permutation analysis.
So at any point in the network, the transformer not only receives information from its past... but also has causal influence over its future processing. So, saying that LLMs cannot introspect... is incorrect.quote0.768
Core summary of Janus' position on autoregressive recurrence enabling introspection.
I think it wouldn’t be unreasonable to have a credence over 50 percent that we’ll have sophisticated LLM+ systems ... within a decade. It also wouldn’t be unreasonable to have at least a 50 percent credence that if we develop sophisticated systems with all of these properties, they will be conscious.quote0.765
David Chalmers' estimate of AI consciousness probability, quoted in §2.2.2.
2/3 Layer of LLMconcept0.764
The layer approximately two-thirds through an LLM's transformer stack, reported to best predict human brain activity; identified as promising for consciousness indicators.