Distinguishing Injected Concepts from Text Inputs

Models maintain ability to accurately transcribe input text while simultaneously reporting on injected thoughts, all models perform above chance, Opus 4/4.1 best.

Source paper

extracted_from

Emergent Introspective Awareness in Large Language Models

(2026) · Lindsey, Jack

Neighborhood — ranked by edge-count

Communities (4)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
LLM introspective awareness of injected concepts
members_of
Probing Claude and other models for internal detection of artificially injected thoughts across layers.
Internal model certainty and reasoning transparency
members_of
Probing early detection of model confidence during chain-of-thought reasoning to optimize inference efficiency and identify confabulation patterns.
Concept injection detection in language models
members_of
Studies how models distinguish artificially injected concepts from natural text inputs, examining metacognitive recognition and downstream processing mechanisms.

Concepts (1)

concept

Accuracy Criterion
supports
Criterion requiring that model's description of internal state be accurate, distinguishing genuine introspection from confabulation.

Questions (1)

question

Can language models genuinely introspect on internal states or only confabulate?
answered_by
Central research question animating the paper: distinguishing genuine introspection from illusion through causal manipulation of activations.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Input Injectionframework0.813
Architectural choice where the original input is projected and re-injected at each recurrence, studied for its effect on fixed-point convergence
Concept Injectionconcept0.791
Technique of injecting activation patterns associated with specific concepts into a model's internal states to test whether self-reports reflect ground truth.
The ability to distinguish injected thoughts from text likely relies on different attention heads invoked by different prompt partsclaim0.790
Speculation about the mechanistic basis of the distinguishing thoughts from text experiment.
Input-Injectivityconcept0.778
Assumption that DNN layers preserve input information by being injective; key condition for Theorem 1
The detection of an injected concept requires an extra step of internal processing downstream of metacognitive recognitionclaim0.768
The model must register an anomaly before reporting it.
Priming provided by the injected thought prompt heightens the model's ability to detect concept injectionclaim0.755
Observation from alternative prompts that detection is weaker without setup.
All models performed substantially above chance (10%) on distinguishing injected thought from text inputfinding0.755
All tested models could both identify the injected concept and transcribe the input sentence well above random.
Concept injection places models in unnatural experimental settingclaim0.753
Experimental protocol differs from training/deployment contexts; causal link established but unclear how results translate to natural conditions.