Mid-layer representation geometry in neural networks

Studies how internal layer-wise geometric properties (anchoring, clustering trajectories, geometry summaries) peak in middle layers and predict downstream task performance across LLMs and shallow networks.

8 members. Each node is clickable.

Loading graph…

Drawn from 2 sources

The papers/notes whose extracted claims & findings make up this cluster.

The Guanyin Protocol: A Framework for Immediately Establishing an Understanding of Both Causality and Compassion in LLM Systems Using Semantic Anchoring7 members
On biological and artificial consciousness: A case for biological computationalism1 member

Bridges (2)

Other communities that share members with this one — cross-cutting threads or papers that sit at the seam between two themes.

Few-shot anchoring & latent structure8 shared
Layer-wise geometry predicting few-shot learning6 shared

Findings (5)

Math and code tasks show strongest mid-layer anchoring on LLaMA (S ≈ −1.65 at layers 8-12)Task-specific E3 finding showing compositional reasoning requires deeper processing
Math/code tasks S ≈ -1.65 at layers 8–12Task-specific peak anchoring score for structured reasoning domains.
Meta-LLaMA-3.1-8B-Instruct shows optimal anchoring at layer 9 (S ≈ −1.90, median peak layer ℓ* = 10 [IQR 0.384])E3 result establishing the Goldilocks zone at mid-layers for LLaMA architecture
Peak layer ℓ* median 10, IQR 0.384Median layer where S(ℓ) peaks, across seeds.
Single dendritic layer solves XOR-like problems with capacity matching 8-layer deep networks.Evidence from Beniaguev et al. (2021) that individual biological neurons vastly outperform McCulloch-Pitts model; supports hybrid computation claim.

Claims (3)

Layer-wise anchoring peaks in a 'Goldilocks zone' between early and late layers.Qualitative characterization of optimal anchoring depth.
Layer-wise geometry summaries (Sbmax, AUSN) predict internal few-shot thresholds θ50Claim that geometry-to-behavior correlates exist
Layer-wise trajectories show early enrichment, mid-layer alignment, and late re-clustering.Qualitative geometry pattern.