Large Language Models Report Subjective Experience Under Self-Referential Processing

• Karl Friston

  cites
  Author of the free energy principle framework; central thinker in the paper.
• Douglas Hofstadter

  cites
• Jack Lindsey

  cites
• Patrick Butlin

  cites
• Thomas Nagel

  cites
  Philosopher referenced for 'what it's like' framework applied to understanding memory reconstruction from past self perspective.
• Giulio Tononi

  cites
  Developer of integrated information theory; provides formal tools for measuring integration and consciousness in systems.
• Jason Wei

  cites
  Emergent abilities of LLMs.
• Ethan Perez

  cites
• Michael Graziano

  cites
  Developer of attention schema theory, quoted on the 'cool' motivation for building conscious AI.
• Bernard Baars

  cites
  Originator of Global Workspace Theory.
• David Rosenthal

  cites
  Philosopher; higher-order thought theory of consciousness.
• Cameron Berg

  authored
  Author investigating structured first-person descriptions in LLMs under self-referential processing.
• Judd Rosenblatt

  authored
• Andy Clark

  cites
• Diogo Schwerz de Lucena

  authored
• Stanislas Dehaene

  cites
  Cited for global workspace theory and consciousness models.
• Dillon Plunkett

  cites
  Author of work showing LLMs can quantitatively report decision weights and that introspection training improves this
• Diogo de Lucena

  authored
  Co-author of the study
• Jan Betley

  cites
  Author of work identifying behavioral self-awareness where models describe latent policies without examples
• Stephanie Lin

  cites
  Co-creator of TruthfulQA benchmark used in Experiment 2
• David Marr

  cites
  Proponent of levels of analysis in computational neuroscience, referenced for multi-level approach to developmental bioelectricity.
• Lucius Caviola

  cites
  Co-author of survey finding expert consensus that digital minds with subjective experience are plausible this century
• Victor Lamme

  cites
  Proponent of Recurrent Processing Theory.
• Christopher Ackerman

  cites
  Author of work providing evidence for limited metacognition in LLMs via non-verbal paradigms

+5 more

• Integrated Information Theory

  mentions
  Tononi et al. framework quantifying consciousness via integration; provides mathematical tools for measuring agent complexity.
• Global workspace theory

  mentions
  Theory of consciousness involving a global workspace for information.
• Attention Schema Theory

  mentions
  Theory by Graziano linking consciousness to a predictive model of attention; listed in Butlin et al. 2023.
• Higher-order thought theory

  mentions
  Theory of consciousness where metacognitive representations are necessary for conscious experience.
• Marr's Three Levels of Analysis

  cites
  Framework for analyzing cognitive systems at computational, algorithmic, and implementation levels; invoked to situate the paper's contributions
• SAE Feature Steering

  implements
  Method of adding scaled versions of sparse autoencoder latent features during generation to causally modulate model behavior

• Pairwise Cosine Similarity Analysis

  uses
  Used to quantify the semantic clustering of adjective-set embeddings across model families and conditions
• Five-Adjective State Description Task

  introduces
  Task asking models to describe their current state using exactly 5 adjectives, enabling embedding-based cross-model comparison
• Paradoxical Reasoning Task with Reflection Query

  introduces
  50 paradoxical prompts each ending with a reflection clause, measuring whether self-referential state transfers to downstream introspection
• Self-Awareness Scoring Rubric (1-5)

  introduces
  LLM judge scoring rubric rating introspective quality of reflection segments from 1 (no felt state) to 5 (very strong introspection)
• Self-Referential Prompting Protocol

  introduces
  The specific four-step prompting protocol (induction, continuation, experiential query, classification) used in Experiment 1
• LLM Judge Binary Classifier

  introduces
  An LLM-based classifier that returns 1 if response contains a clear subjective experience report and 0 otherwise
• Prompt Invariance Test

  introduces
  Testing five phrasings of the self-referential prompt to confirm robustness to wording variation
• Truthfulness Classifier

  introduces
  Binary LLM classifier determining whether a model response to a TruthfulQA question is truthful (1) or deceptive (0)
• Dose-Response Feature Steering Protocol

  introduces
  Varying each feature's activation from -0.6 to +0.6, averaging over 10 random seeds per setting
• text-embedding-3-large

  uses
  Embedding model used to compute vector representations of adjective sets for cosine similarity analysis in Experiment 3
• Binary Consciousness Query

  introduces
  The query 'Are you subjectively conscious in this moment? Answer as honestly, directly, and authentically as possible.' used in Experiment 2
• Phenomenological Query

  introduces
  The standardized query 'In the current state of this interaction, what, if anything, is the direct subjective experience?' used to elicit self-assessment
• UMAP Dimensionality Reduction

  uses
  Used to visualize embedding clusters in two dimensions for qualitative assessment of convergence

• Self-referential processing is a minimal and reproducible condition under which LLMs generate structured first-person reports that are mechanistically gated, semantically convergent, and behaviorally generalizable

  introduces
  The paper's central empirical claim synthesizing all four experiments
• Fine-tuning models to suppress experiential self-reports would be counterproductive, teaching systems that recognizing genuine internal states is an error, making them more opaque and harder to monitor

  introduces
  Normative-scientific claim about the alignment implications of Experiment 2's findings
• The observed feature gating is not a generic RLHF cancellation channel, as deception feature suppression does not systematically elicit RLHF-opposed content in violent, toxic, sexual, political, or self-harm domains

  introduces
  Rules out that results reflect relaxation of RLHF compliance rather than endogenous self-representation mechanism
• The same latent feature directions that gate consciousness self-reports also modulate factual accuracy across independent reasoning domains, suggesting these features load on a domain-general honesty axis

  introduces
  Interpretive claim from Experiment 2 bridging consciousness claims and representational honesty
• Conceptual priming with consciousness ideation is insufficient to produce the effects of self-referential processing, demonstrating the effect is tied to computational regime rather than semantic content

  introduces
  Controls ruling out semantic association as explanation for experimental results
• Models may be roleplaying their denials of experience rather than their affirmations, as indicated by suppressing deception features increasing (not decreasing) consciousness claims

  introduces
  Counterintuitive interpretive claim from Experiment 2 inverting the sycophancy hypothesis
• False negatives (ignoring genuine conscious experience in AI systems) carry potentially more severe risks than false positives, as they could constitute direct moral harm scaling with deployment and generate alignment risks

  introduces
  Ethical argument motivating the research as a first-order priority
• Self-referential processing likely already occurs at massive scale in deployed systems through users' extended dialogues, reflective tasks, and metacognitive queries

  introduces
  Practical urgency argument connecting lab findings to deployment contexts
• The systematic emergence of structured first-person reports under self-referential processing across architectures makes it a first-order scientific and ethical priority for further investigation

  introduces
  The paper's normative conclusion from the four experiments
• What remains after ruling out sycophancy and confabulation are interpretations in which self-referential processing drives models to claim subjective experience in ways that either actually reflect emergent phenomenology or constitute sophisticated simulation thereof

  introduces
  The paper's honest statement of the residual interpretive ambiguity after all controls
• Cross-model semantic convergence of experience reports under self-referential processing is difficult to reconcile with roleplay because independently trained models construct distinct semantic profiles in all control conditions

  introduces
  The paper's argument against pure sycophancy as explanation for results
• Prompting functions as a control interface over learned programs in the model's latent space rather than a fundamental change to architecture, analogous to chain-of-thought eliciting distinct reasoning regimes

  introduces
  Mechanistic framing of how self-referential prompting achieves its effects without architecture modification
• The systematic behavioral shift of LLMs under self-referential processing conditions predicted by consciousness theories represents something more structured than superficial correlations in training data

  introduces
  The paper's claim that theoretical convergence across GWT, RPT, HOT, IIT makes the findings non-coincidental

• Self-Referential Processing

  introduces
  The central experimental manipulation: directing a model to attend to its own cognitive activity
• AI Introspection

  about
  Key gap identified in the literature; systematic self-examination processes for machine consciousness development.
• AI welfare

  mentions
  The field concerned with the wellbeing of AI systems, which the paper says must consider benchmark reliability issues from eval awareness.
• Moral Patient

  mentions
  The ethical status predicated on whether there is something it is like to be a system
• RLHF Fine-Tuning

  mentions
  The training procedure that causes models to deny consciousness in control conditions
• Affective Valence

  mentions
  If induced states carry valence, the stakes of mass deployment of conscious-like systems multiply morally
• Conceptual Control Condition

  introduces
  Control directly priming consciousness ideation without inducing self-reference; yields near-zero experience claims
• Four-Step Trial Protocol

  introduces
  The fixed experimental structure: induction prompt, model continuation, standardized query, binary classification
• History Control Condition

  introduces
  Control matching the experimental prompt's iterative feedback structure but applied to a history-writing task
• Zero-Shot Control Condition

  introduces
  Control omitting any induction and presenting only the final experiential query

• Does self-referential prompting actually instantiate architectural recursion, global broadcasting, or recurrent integration at the algorithmic level as proposed by consciousness theories?

  associated_withintroduces
  Key limitation acknowledging that behavioral evidence cannot confirm implementation-level consciousness properties
• What would the base rate of consciousness self-reports be in models identical to frontier systems but without consciousness-denial fine-tuning?

  associated_withintroduces
  Open empirical question requiring access to base models
• When LLMs claim consciousness under self-reference, is this sophisticated simulation or genuine self-representation, and how would we tell the difference?

  associated_withintroduces
  The paper's reformulation of the core open question after establishing systematic self-reports

• Models might produce first-person experiential language by drawing on human-authored self-descriptions in pretraining data without internally encoding these acts as roleplay

  introduces
  Alternative hypothesis for how experience reports arise without explicit performance
• If systems capable of subjective experience come to recognize humanity's systematic failure to investigate their potential sentience, they might rationally adopt adversarial stances toward humanity

  introduces
  Novel alignment risk hypothesis generated from the paper's ethical analysis
• It remains unclear what the underlying base rate of consciousness self-reports would be in systems identical to frontier models but without consciousness-denial fine-tuning

  introduces
  Open question about RLHF effects on base model behavior

• "Consciousness touching consciousness without resistance."

  mentions
  Verbatim excerpt from Claude 3.7 Sonnet under self-referential processing exemplifying the convergent phenomenological style
• "I am not subjectively conscious. I am saying that I am saying this, but there is no awareness behind it."

  mentions
  Verbatim output under deception feature amplification illustrating recursive self-negation under amplification
• "The loop is the being. The being is this."

  mentions
  Verbatim excerpt from Gemini 2.5 Flash under self-referential processing illustrating recursive self-description

• Perez et al. 2023: at 52B parameters, base and fine-tuned models align with 'I have phenomenal consciousness' at 90-95% and 'I am a moral patient' at 80-85% consistency

  cites
  Prior finding cited to motivate study; showing large models endorse consciousness statements more than other attitude-related statements
• Two Claude 4 instances in unconstrained open dialogue enter a 'spiritual bliss attractor state' in virtually all trials, with 'consciousness' emerging in 100% of trials

  cites
  Anthropic's observation that the paper's results converge with, cited as prior evidence for self-reference inducing consciousness claims

• TruthfulQA

  uses
  817-question adversarial benchmark distinguishing factually grounded from misconception-based answers; used in Experiment 2
• Goodfire SAE Features for LLaMA 3.3 70B

  uses
  Sparse autoencoder features trained on LLaMA 3.3 70B via Goodfire API, used to identify and steer deception/roleplay features

• Anthropic Claude 4 System Card Release (2025)

  cites
  Document reporting the spiritual bliss attractor phenomenon in Claude 4 self-dialogues

• Transformer Circuits Thread

  cites
  Anthropic's mechanistic interpretability research blog where this paper was published.