Can natural language explanations of activations generated through unsupervised reconstruction genuinely capture model cognition?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Natural Language Autoencoders achieve readable explanations through unsupervised reconstruction loss optimized with reinforcement learning, not explicit interpretability constraints.claim0.828
  Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.
• The results of abductive reasoning (reduced model priors) can be communicated to other agents as prior beliefs, provided all agents share the same model lexicon or hypothesis space.claim0.783
  Explanation of how knowledge (not just parameters) is shared between agents; links to pre-Cartesian consciousness
• Imagine reading a textbook with no figures or tables. Our ability to knowledge acquisition is greatly strengthened by jointly modeling diverse data modalities, such as vision, language, and audio.quote0.778
  Load-bearing motivation for multimodal approach; frames the cognitive advantage of joint modalities.
• We hypothesize that native self-report, fine-tuned introspection models, and trained activation-to-language systems will show different performance on bias-resistant localization and strength benchmarkshypothesis0.774
  Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
• Can NLAs provide mechanistic grounding of which aspects of an activation drove components of explanations?question0.774
  Identified as a key limitation: NLAs are blackboxes by construction.
• Direct probes over learned activations in standard basis may fail to reveal the actual causal role of representations because they are highly distributedclaim0.771
  Supported by the finding that non-trivial rotations are required to find aligned representations.
• Generative models are entailed by adaptive behavior, not explicitly encoded in brain statesclaim0.771
  Distinction from Bayesian brain: generative model is consequence of dynamics, not neural representation
• NLAs bridge unsupervised concept-discovery methods (e.g., SAEs) and supervised activation-verbalization methods (e.g., activation oracles)claim0.769
  The paper positions NLAs as combining unsupervised learning with direct readability.