Layer-dependent introspective peaks

Introspective awareness in Opus 4.1 peaks at layer ~2/3 through model depth for thought injection and text distinction; prefill detection most sensitive to earlier layer, suggesting mechanistically distinct processes.

Source paper

extracted_from

Emergent Introspective Awareness in Large Language Models

(2026) · Lindsey, Jack

Neighborhood — ranked by edge-count

Communities (4)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic introspection in language models
members_of
Empirical investigation of how LMs access and report internal states across layers, using concept injection and thought detection on Claude models.
LLM functional introspective awareness
members_of
Empirical probing of language models' ability to detect and report their own internal concept representations
Introspective signal suppression in transformer layers
members_of
Study of how self-referential representations emerge in middle layers but are actively dampened by post-training procedures like RLHF.

Questions (1)

question

What are the mechanistic bases of introspective awareness in LLMs?
gates
Secondary question; paper demonstrates introspection but explicitly avoids pinning down specific mechanistic explanation, noting mechanisms could be shallow and specialized.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Computational Account of Layer-Dependent Introspectionframework0.829
This paper's proposed mechanistic explanation integrating signal injection, attention routing, predictive integration, and residual recovery
Introspective signals appear in middle layers but are suppressed by later post-training-shaped layers.finding0.783
Mechanistic finding by Lindsey (2026) explaining how contemplative prompt may work: enables mid-layer introspection to reach output.
Introspective awareness peaks at a layer about two-thirds through Opus 4.1 for injected thoughtsfinding0.765
The success rate shows a sharp peak at a specific middle layer.
Introspective awarenessconcept0.731
The central concept: the ability of a model to access and report on its internal states, as defined by the paper's criteria.
Introspective capabilities are confined to early-layer injections (L0-L5) and collapse to chance thereafterclaim0.731
Key quantitative characterization of the layer-dependence of partial introspection
Layer-wise anchoring peaks in a 'Goldilocks zone' between early and late layers.claim0.726
Qualitative characterization of optimal anchoring depth.
Introspection Stagemethod0.721
Stage 3 of character training: SFT on synthetic introspective data generated by post-distillation checkpoint
LLMs can compute meaningful functions over perturbations to their internal states, establishing introspection as a real but layer-dependent phenomenonclaim0.720
Primary positive claim of the paper, grounded in strength comparison and localization results