claim
active
claim:the-identification-of-reasoning-steps-relies-on-keyword-search-which-may-be-model-specific-since-different-models-could-prefer-different-reflection-cuesThe identification of reasoning steps relies on keyword search, which may be model-specific since different models could prefer different reflection cues
Limitation acknowledged regarding generalizability of the reflection identification method
Source paper
extracted_from(2025) · Ge Yan · Sun, Chung-En · Tsui-Wei · Weng
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Method to identify reflection steps by searching for specific keywords (e.g., 'Let me think', 'Wait') within reasoning steps
- Gap in current evaluation methods; current work relies on CoT monitoring which may miss unverbalized beliefs.
- Opus 4.1 demonstrates highest introspective awareness on abstract nouns (justice, peace, betrayal) with nonzero awareness across all concept categories tested.
- All models exhibit above-baseline representation of the think word when instructed to think about itfinding0.767In the intentional control experiment, all tested models show above-zero cosine similarity to the think word's concept vector.
- Empirical gap explicitly acknowledged; experiments reportedly in progress at time of writing
- Acknowledges that the model's additional descriptions of its experience are unverified.
- Core definitional quote for performative chain-of-thought
- Claim that capability emerges from architecture, not data, and that later models lose the surprise.