Detecting Unintended Outputs via Introspection

Models can distinguish artificially prefilled outputs from intentional responses by referencing prior internal representations; injection of matching concept vector causes model to retroactively accept prefill as intentional.

Source paper

extracted_from

Emergent Introspective Awareness in Large Language Models

(2026) · Lindsey, Jack

Neighborhood — ranked by edge-count

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic introspection in language models
members_of
Empirical investigation of how LMs access and report internal states across layers, using concept injection and thought detection on Claude models.
Model introspection for misalignment detection
members_of
Using architectural self-knowledge prompts to improve models' ability to identify their own unintended outputs.

Concepts (1)

concept

Internality Criterion
supports
Criterion requiring that causal influence of internal state on description be internal, not routed through sampled outputs; rules out pseudo-introspection via self-observation.

Questions (1)

question

Can language models genuinely introspect on internal states or only confabulate?
answered_by
Central research question animating the paper: distinguishing genuine introspection from illusion through causal manipulation of activations.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

we operationalize introspection as causal informational coupling between a numeric self-report and an independently measured internal directionquote0.793
Load-bearing operational definition that distinguishes the paper's framework from prior approaches
Introspection relies on general-purpose computational mechanisms—attention-based anomaly detection and residual stream dynamics—rather than specialized introspection circuitsclaim0.778
Interpretive claim about the mechanistic substrate of introspection in LLMs
Self-Report Method for AI Introspectionmethod0.766
Technique of eliciting and interpreting AI self-reports to assess internal states; discussed as promising but challenging.
Either introspection is an emergent capability requiring larger scale, or more stringent controls are needed to test introspection in smaller modelsclaim0.760
Alternative interpretations offered for why binary detection fails in Llama 3.1 8B but frontier models claim success
unpredictability of end-resultconcept0.757
The principle that a truly adaptive process cannot have a predetermined end-state; adaptation means changes cannot be foreseen.
Introspective ability can be decomposed into: (i) information available about internal state and (ii) capacity to transform that signal into precise output reportsclaim0.756
Conceptual distinction motivated by entropy analyses showing probe and report entropy can diverge under steering
Introspective awarenessconcept0.751
The central concept: the ability of a model to access and report on its internal states, as defined by the paper's criteria.
Observed introspection may lack philosophical significance of human introspectionclaim0.751
Paper does not address whether AI introspection constitutes self-awareness or subjective experience; mechanistic uncertainty prevents definitive philosophical claims.