finding

active

finding:systematic-layer-20-28-degradation-in-s-l-to-s-2-40-by-layer-27-on-llama

Systematic layer 20-28 degradation in S(ℓ) to S ≈ −2.40 by layer 27 on LLaMA

Validates representational drift theory: later layers specialize for next-token prediction, increasing dr

Source paper

extracted_from

The Guanyin Protocol: A Framework for Immediately Establishing an Understanding of Both Causality and Compassion in LLM Systems Using Semantic Anchoring

(2025) · Edward Yi Chang · Kaya, Zeyneb N. · Ethan Chang

Neighborhood — ranked by edge-count

Concepts (1)

concept

representational drift
associated_withsupports
Accumulation of mismatch in later layers causing S degradation.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Layer 24 (indexed at 8) of LLaMA3.1-8B on Hinting satisfies Criteria 1 and 2 under both IIT 3.0 and IIT 4.0 (temporal permutation).finding0.805
One of the most promising cases; approximately corresponds to the 2/3 layer of LLaMA3.1-8B.
LLaMA-3.1-8B: Sbmax = -1.896 ± 0.211, AUSN = -2.119 ± 0.198, peak layer ℓ* = 10 (median)finding0.798
Seed-pooled geometry-only statistics (per-dev z units).
Math and code tasks show strongest mid-layer anchoring on LLaMA (S ≈ −1.65 at layers 8-12)finding0.796
Task-specific E3 finding showing compositional reasoning requires deeper processing
Layer 29 (indexed at 10) of LLaMA3.1-8B on Strange Stories (2 scores) satisfies Criteria 1 and 2 under IIT 4.0 (temporal permutation).finding0.794
Third promising case from temporal permutation analysis.
Correlation between layer-wise scores and task accuracy ρ = −0.73 (p < 0.001) on LLaMAfinding0.791
Core E3 finding validating S as a predictor of anchoring effectiveness
The case at approximately the 2/3 layer of LLaMA3.1-8B (Layer 24, satisfying Criteria 1 and 2) aligns with prior studies showing the 2/3 layer optimally predicts human brain activity.finding0.787
Connects this study's results to Schrimpf et al. 2021 and Caucheteux et al. 2022/2023 findings on brain-LLM alignment.
LLaMA E3 geometry summary: S_max = −1.896 ± 0.211, AUS_N = −2.119 ± 0.198, peak layer ℓ* = 10 [IQR 0.384]finding0.780
Seed-pooled geometry statistics for LLaMA in E3, providing quantitative basis for geometry-to-behavior correlate
Mean difference patching on Llama-3-8B layer 10 produces intervened EMD exceeding the natural-natural baselinefinding0.773
Empirical demonstration that MDVP produces divergent representations in a real LLM