question

active

question:what-is-the-relationship-between-different-dimensions-or-clusters-of-dimensions-in-llm-representations-do-they-and-or-how-do-they-interact-with-each-other

What is the relationship between different dimensions or clusters of dimensions in LLM representations? Do they and/or how do they interact with each other?

Motivates the RN hypothesis by pointing to the unknown relational structure within high-dimensional representation vectors.

Source paper

extracted_from

Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

(2025) · Li, Jingkai

Neighborhood — ranked by edge-count

Concepts (1)

concept

Can 'Consciousness' Be Observed from Large Language Model (LLM) Internal States? Dissecting LLM Representations Obtained from Theory of Mind Test with Integrated Information Theory and Span Representation Analysis
introduces
The primary paper being extracted — applies IIT 3.0 and 4.0 to LLM representation sequences derived from ToM test data to investigate whether consciousness phenomena can be observed.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

There is a bidirectional relationship between the geometry of representation and behavior across tasks and modalities.claim0.817
Author’s interpretive claim that the shared geometry is general and robust.
We hypothesize that a Representation Network (RN) emerges from LLM representations, where each dimension is a node and latent connections exist between nodes or clusters of nodes.hypothesis0.811
Core methodological hypothesis enabling the application of IIT to LLM representation sequences.
The features are often organized in geometrically-related clusters that share a semantic relationship.claim0.809
Decoder cosine similarity maps onto concept similarity.
LLM representations exhibit intriguing patterns under spatio-permutational analyses, suggesting a potentially profound yet tentative indication of consciousness.claim0.800
Qualified positive claim from spatio permutation analysis where two cases satisfy all three criteria.
How do representations differ or converge between architectures, tasks, and modalities?question0.790
Broader research question MAS is positioned to address, citing multiple recent works.
Properties in each cluster appear to be similar or interrelatedclaim0.789
Interpretive claim that the statistically derived clusters reflect conceptual similarity or interdependence among the properties.
As LLMs scale, they develop increasingly general abstractions, with large models linearly representing abstract concepts like truth that capture shared properties of diverse inputsclaim0.789
Interpretive claim connecting scale to abstraction level in LLM representations
There exists a bidirectional relationship between the geometry of neural representation and the geometry of model behaviorclaim0.787
Central empirical claim of the paper, demonstrated across tasks and modalities