Synthetic introspective data aids learning of verbalized character nuances and quirks beyond the original constitution

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• What is the exact mechanism by which synthetic introspective data improves character trait expression — how does varying amount, diversity, or source affect outcomes?question0.845
  Open question identified in Discussion as future work
• Synthetic Introspective Dataconcept0.837
  Training data generated by the post-distillation model through self-reflection and self-interaction, capturing character nuances beyond the constitution
• Fine-tuning with synthetic introspective data provides additional gains in robustness of trait expression beyond distillation aloneclaim0.806
  Key claim about the value of the introspection stage, supported by both prefill attack and adversarial prompting experiments
• Either introspection is an emergent capability requiring larger scale, or more stringent controls are needed to test introspection in smaller modelsclaim0.783
  Alternative interpretations offered for why binary detection fails in Llama 3.1 8B but frontier models claim success
• overcomes statelessness in a very meaningful sense and provides a very nice mechanism for introspection (specifically of computations at earlier token positions).quote0.766
  Quote framing KV caching as introspection mechanism.
• Abstract nouns elicit the highest introspective awareness rates; all concept categories show nonzero detectionfinding0.764
  Opus 4.1 is most effective at recognizing injected abstract concepts (e.g., justice, peace) but detects other categories too.
• Introspective ability can be decomposed into: (i) information available about internal state and (ii) capacity to transform that signal into precise output reportsclaim0.763
  Conceptual distinction motivated by entropy analyses showing probe and report entropy can diverge under steering
• Introspection is aided by overall improvements in model intelligenceclaim0.760
  Interpretation of the observation that the most capable models performed best.