finding

active

finding:five-prediction-tasks-improve-with-nla-training-across-three-models-opus-4-6-haiku-4-5-haiku-3-5

Five prediction tasks improve with NLA training across three models (Opus 4.6, Haiku 4.5, Haiku 3.5).

Systematic evidence that NLA explanations become more informative during training despite optimizing only for reconstruction.

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Claims (1)

claim

Natural Language Autoencoders achieve readable explanations through unsupervised reconstruction loss optimized with reinforcement learning, not explicit interpretability constraints.
supports
Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.

Communities (2)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Natural Language Autoencoders for mechanistic interpretability
members_of
Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

There are fewer representations competent for N tasks than M<N tasks, so training more general models should yield fewer possible solutionshypothesis0.776
Selective pressure toward convergence via task generality
NLA explanations grow more informative over training with FVE increasing from 0.3-0.4 to 0.6-0.8 roughly linearly in log(training steps)finding0.776
Quantitative evidence that NLA training produces increasingly informative explanations despite optimizing only for reconstruction.
Notably, Claude Opus 4.1 and 4—the most recently released and most capable models of those that we test—perform the best in our experiments, suggesting that introspective capabilities may emerge alongside other improvements to language models.quote0.768
Key finding about the relationship between capability and introspection.
Training on image data should improve LLM performance, and training on language data should improve vision model performancehypothesis0.760
Implication of PRH for cross-modal training efficiency
Ouyang et al. 2022: Training language models to follow instructions with human feedbackconcept0.752
RLHF paper cited as a major fine-tuning technique used in commercial dialogue agents
NLA-derived steering vectors from edited explanations can causally shift planning representations, changing rhyme completion from 'rabbit' to 'mouse' at ~50% success rate.finding0.745
Evidence that NLA explanations bear causal relationship to model outputs; demonstrates validity of extracted representations.
For both NPI and filler-gap tasks, the model initially learns to move information directly from alternating token to output; intermediate steps are added later in trainingclaim0.744
Mechanistic interpretation of training dynamics in case studies
SL-CAI training with up to 4 revisions improves harmlessness; SL-CAI-n models are trained with n revisions, n=1,2,3,4.finding0.739
Section 3.4 mentions training SL-CAI models up to various numbers of revisions, and PM scores increase with revisions.