finding

active

finding:nla-explanations-grow-more-informative-over-training-with-fve-increasing-from-0-3-0-4-to-0-6-0-8-roughly-linearly-in-log-training-steps

NLA explanations grow more informative over training with FVE increasing from 0.3-0.4 to 0.6-0.8 roughly linearly in log(training steps)

Quantitative evidence that NLA training produces increasingly informative explanations despite optimizing only for reconstruction.

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Claims (1)

claim

NLAs bridge unsupervised concept-discovery methods (e.g., SAEs) and supervised activation-verbalization methods (e.g., activation oracles)
supports
The paper positions NLAs as combining unsupervised learning with direct readability.

Communities (2)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Natural Language Autoencoders for mechanistic interpretability
members_of
Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Five prediction tasks improve with NLA training across three models (Opus 4.6, Haiku 4.5, Haiku 3.5).finding0.776
Systematic evidence that NLA explanations become more informative during training despite optimizing only for reconstruction.
NLA explanations confabulate false specifics but maintain thematic fidelity; claims repeated across tokens more likely true than isolated claims.claim0.773
Core limitation and usage heuristic: read NLAs for themes rather than individual factual claims; cross-check with original context.
NLA explanations can contain claims about the target model's input context that are verifiably false, but are typically thematically faithful to the context.claim0.766
Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.
Little evidence of steganography in NLAs; meaning-preserving transformations cause only small drops in FVEfinding0.760
Quantitative evaluation showing NLAs do not heavily rely on covert encoding beyond overt language.
Editing NLA explanations to change 'reward' to 'penalty' produces steering vector that increases odd-number responses from near-zero to >70%, demonstrating belief capture upstream of behavior.finding0.755
Shows NLA explanations capture latent model beliefs about rewards before output selection; validates interpretability.
The generalization improvement from explicit instructions observed in Llama models (A1-A3 to F0-F2) is more pronounced for F3-F5 to F0-F2 in Gemma models.claim0.752
Shows the instruction effect, while shifting geometry, may not produce consistent generalization effects across model families.
DialoGPT on DailyDialog++: NLI Diversity increases from 4.11 to 10.24 with 6.3 samplesfinding0.746
DTG result for DialoGPT on DailyDialog++ using NLI metric
NLA-derived steering vectors from edited explanations can causally shift planning representations, changing rhyme completion from 'rabbit' to 'mouse' at ~50% success rate.finding0.745
Evidence that NLA explanations bear causal relationship to model outputs; demonstrates validity of extracted representations.