claim

active

claim:pca-is-the-appropriate-dimensionality-reduction-technique-for-constructing-the-rn-because-it-preserves-global-structure-and-provides-deterministic-interpretable-projections

PCA is the appropriate dimensionality reduction technique for constructing the RN because it preserves global structure and provides deterministic, interpretable projections.

Justifies PCA choice over UMAP or t-SNE for the node-structured RN model.

Source paper

extracted_from

Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis

(2025) · Li, Jingkai

Neighborhood — ranked by edge-count

Concepts (1)

concept

Can 'Consciousness' Be Observed from Large Language Model (LLM) Internal States? Dissecting LLM Representations Obtained from Theory of Mind Test with Integrated Information Theory and Span Representation Analysis
introduces
The primary paper being extracted — applies IIT 3.0 and 4.0 to LLM representation sequences derived from ToM test data to investigate whether consciousness phenomena can be observed.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Principal components analysis (PCA)method0.816
Statistical method used to analyze neural activity data.
PCA Visualizationmethod0.767
Used to visually inspect separation of truth-related directions in model activation space across layers
PCA analysis shows token embeddings and unembeddings are concentrated in a relatively small fraction of residual stream dimensions in large modelsfinding0.765
Supporting evidence for the claim that most residual stream dimensions are free for other layers to use
Functional valence is used for dimensionality reduction when multiple parallel processing paths interfere.claim0.753
Novel claim by Antra, linking valence to computational efficiency in transformers.
FCA offers methods to split large diagrams, browse through lattices, and coarsen views while preserving information.claim0.749
Description of FCA capabilities for handling large data sets.
PCA visualizations of LLaMA-2-13B and 70B representations of curated datasets show clear linear structure, with true statements separating from false ones in the top two principal componentsfinding0.745
Primary visual evidence for linear truth representations in large LLMs
Scaled dot-product attention is the most faithful, structured, and theoretically grounded method for incorporating stimulus influence into response representations leading to an RN.claim0.733
Justifies the methodological choice of attention over concatenation, mean pooling, residual connections, or joint embedding.
In LLaMA-2-7B, PCA of larger_than+smaller_than shows statements clustering by surface-level characteristics (e.g., presence of token 'eighty') rather than truth valuefinding0.731
Shows absence of abstract truth representations in smallest model, supporting scale-dependent emergence claim