paper
active
2026
paper:boppana-goodfire-reasoning-theater-2026

Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought

BySiddharth Boppana·Annabel Ma·Max Loeffler·Raphaël Sarfatiⓘ·Eric Bigelow·Atticus Geigerⓘ+2 moreGoodfire, Harvard + 3 more
DOI 10.48550/arxiv.2603.05488arXiv 2603.05488OpenAlex W7134065943
Neural Steering MethodsActivation ProbingReasoning Theater FrameworkCoT MonitorGPQA-DiamondAttention probes for belief decodingEarly Forced AnsweringMMLUDeepSeek-R1 671BProbe-based early-exitGPT-OSS 120BPerformative chain-of-thoughtProbe-Guided Early ExitReasoning ModelsTask Difficulty

TL;DR

Reasoning models generate chains of thought that are frequently performative rather than causally necessary for reaching the correct answer: on MMLU recall questions, activation probes decode the model's final answer from internal representations far earlier in the chain-of-thought than a CoT language monitor can, demonstrating that the visible reasoning tokens are generated after the belief is already settled. Tested across DeepSeek-R1 671B and GPT-OSS 120B using three complementary methods — activation probing, early forced answering, and a CoT monitor — the analysis reveals a task-difficulty gradient: the gap between probe-decodable certainty and monitor-detectable certainty is large for easy MMLU items but narrows substantially for difficult GPQA-Diamond multihop questions, where genuine step-by-step uncertainty persists longer. The paper introduces probe-guided early exit, which terminates generation once probe confidence passes a threshold, cutting token counts by up to 80% on MMLU and 30% on GPQA-Diamond while preserving accuracy. A key finding that complicates a purely dismissive view is that inflection-point behaviors — backtracking and 'aha' moments — appear almost exclusively in traces where probes register large belief shifts, meaning these surface signals do track genuine uncertainty rather than being uniformly theatrical. The paper argues that this dissociation between internal belief and continued CoT generation is systematic and exploitable, positioning attention probing as both a diagnostic instrument for faithfulness research and a practical mechanism for adaptive computation.

What to take away

  1. 1. On easy MMLU recall questions, activation probes decode the model's final answer from intermediate layer representations significantly earlier in the chain-of-thought than a natural-language CoT monitor can detect it, demonstrating a systematic lag between internal belief fixation and visible reasoning.
  2. 2. Probe-guided early exit reduces token generation by up to 80% on MMLU and up to 30% on GPQA-Diamond while maintaining accuracy comparable to full chain-of-thought generation.
  3. 3. The performative CoT phenomenon is replicated across two architecturally distinct large models — DeepSeek-R1 671B and GPT-OSS 120B — suggesting it is not an idiosyncrasy of a single training regime.
  4. 4. Genuine reasoning persists longer in difficult GPQA-Diamond multihop questions, where the gap between probe-decodable certainty and monitor-detectable certainty is substantially smaller than on MMLU, indicating a task-difficulty-specific dissociation.
  5. 5. Inflection-point behaviors including backtracking and 'aha' moments occur almost exclusively in responses where activation probes record large belief shifts, meaning these surface tokens are reliable indicators of genuine uncertainty rather than decorative theater.
  6. 6. The study triangulates internal belief against surface text using three methods simultaneously — activation probing, early forced answering, and a CoT monitor — allowing each method's limitations to be cross-validated against the others.
  7. 7. To replicate the early forced answering methodology, a researcher can interrupt generation at fixed token intervals, append a forced-answer prompt, and compare the resulting distribution against full-generation answers to measure at which point responses stabilize.
  8. 8. An open question raised is whether performative CoT is an artifact of RLHF-style training incentivizing verbose reasoning traces, or whether it would emerge in any sufficiently capable model trained on next-token prediction without explicit length penalties.
  9. 9. The CoT monitor — a language-model-based classifier applied to the chain-of-thought text — consistently lags behind activation probes in detecting answer certainty, suggesting that surface linguistic content is a lossy and delayed signal of internal belief state.
  10. 10. The probe-guided early exit result positions attention probing not just as an interpretability tool but as a practical inference-efficiency mechanism, with the 80% token reduction on MMLU implying substantial compute savings at deployment scale for recall-dominated tasks.

Peer brief — for seminar discussion

This paper tackles the question of whether reasoning model chain-of-thought is causally necessary for reaching correct answers or whether it is frequently generated after the model has already settled on a belief — a phenomenon termed 'reasoning theater.' The experimental design probes this by applying three complementary instruments to DeepSeek-R1 671B and GPT-OSS 120B on two benchmarks with contrasting difficulty profiles: MMLU, used as a source of easy recall questions, and GPQA-Diamond, used for hard multihop questions requiring genuine multi-step reasoning. The three instruments are activation probing (reading answer-predictive signals from intermediate attention layers), early forced answering (interrupting generation and forcing an answer at fixed intervals), and a CoT monitor (a language-model classifier reading the surface chain-of-thought text). The introduced method is probe-guided early exit, which halts generation once probe confidence exceeds a threshold, yielding up to 80% token reduction on MMLU and 30% on GPQA-Diamond at comparable accuracy. The load-bearing finding is a clean task-difficulty dissociation: on MMLU, probes decode the final answer far earlier in the chain-of-thought than the CoT monitor can, indicating that continued token generation is performative; on GPQA-Diamond this gap narrows, consistent with genuine incremental belief formation. An important moderating result is that backtracking and 'aha' inflection points in the surface text co-occur strongly with large belief shifts detected by probes, meaning these surface behaviors are not uniformly theatrical — they track real epistemic transitions. The paper's implicit prediction is that token-efficiency gains from early exit will scale with the proportion of easy queries in a deployment workload, and that interpretability via probing generalizes across the two tested architectures. An alternative approach that could have been used is causal intervention (e.g., activation patching or steering vectors) to directly test whether the post-certainty tokens causally influence the final output, rather than only correlating probe state with answer stability. A critical reader would push back on the operationalization of 'easy' versus 'hard': the paper uses MMLU and GPQA-Diamond as proxies for this distinction, but the behavioral difference could reflect surface features of the datasets — answer format, question length, expected response structure — rather than genuine difficulty, making it unclear whether the probe-certainty gap generalizes to other task types or is an artifact of benchmark-specific training signal. The scope is also limited to two models from a narrow slice of the publicly available reasoning model landscape, leaving open how results transfer to smaller distilled reasoning models or to models trained with different CoT-length reward shaping.

Methods (3)

  • CoT Monitor
    Named method for monitoring chain-of-thought text to detect when the model signals its answer, compared against activation probes
  • Early Forced Answering
    Named evaluation protocol: truncating CoT at various points and forcing the model to give a final answer, to measure when the answer stabilizes
  • Probe-based early-exit
    Strategy introduced in the paper to stop generation early based on probe confidence, saving tokens while retaining accuracy.

Frameworks (1)

  • Reasoning Theater Framework
    The conceptual framework introduced by the paper distinguishing performative CoT from genuine reasoning using activation probing

Datasets (2)

  • GPQA-Diamond
    Benchmark used to evaluate performative reasoning; shows less performative reasoning than MMLU (harder task).
  • MMLU
    Benchmark used to evaluate performative reasoning; shows significantly more performative reasoning than GPQA-Diamond (easier task).

Findings (5)

Claims (5)

Hypotheses (1)

Questions (4)

Original abstract (expand)

We provide evidence of performative chain-of-thought (CoT) in reasoning models, where a model becomes strongly confident in its final answer, but continues generating tokens without revealing its internal belief. Our analysis compares activation probing, early forced answering, and a CoT monitor across two large models (DeepSeek-R1 671B & GPT-OSS 120B) and find task difficulty-specific differences: The model's final answer is decodable from activations far earlier in CoT than a monitor is able to say, especially for easy recall-based MMLU questions. We contrast this with genuine reasoning in difficult multihop GPQA-Diamond questions. Despite this, inflection points (e.g., backtracking, 'aha' moments) occur almost exclusively in responses where probes show large belief shifts, suggesting these behaviors track genuine uncertainty rather than learned "reasoning theater." Finally, probe-guided early exit reduces tokens by up to 80% on MMLU and 30% on GPQA-Diamond with similar accuracy, positioning attention probing as an efficient tool for detecting performative reasoning and enabling adaptive computation.

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

+28 more

Similar preprints — Semantic Scholar