Natural Language Autoencoders (NLAs)

Core unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained with RL.

Neighborhood — ranked by edge-count

Findings (6)

finding

Model precomputes answers before tool invocation and attends to cached answer over tool output when discrepancy exists, confirmed via attribution graphs.
supports
Mechanistic insight surfaced by NLA explanations and validated through independent causal attribution method.
Editing NLA explanations to change 'reward' to 'penalty' produces steering vector that increases odd-number responses from near-zero to >70%, demonstrating belief capture upstream of behavior.
supports
Shows NLA explanations capture latent model beliefs about rewards before output selection; validates interpretability.
NLA-derived steering vectors from edited explanations can causally shift planning representations, changing rhyme completion from 'rabbit' to 'mouse' at ~50% success rate.
supports
Evidence that NLA explanations bear causal relationship to model outputs; demonstrates validity of extracted representations.
Automated auditing benchmark requiring end-to-end investigation of intentionally-misaligned model; NLA-equipped agents outperform baselines.
supports
Downstream task validating NLA utility for model auditing; agents succeed without access to misalignment training data.
Language switching caused by malformed training data—model fixates on spurious cues inferring user's non-native status, detected via NLA representations preceding foreign-language output.
supports
Case study demonstrating NLA ability to surface root causes of model misbehavior; corroborated by training data inspection.
NLA-equipped auditing agents outperform baselines on misalignment investigation task.
supports
Demonstrates practical utility: NLAs enable root-cause discovery without access to misaligned model's training data.

Methods (6)

method

Activation Oracles (AO)
associated_withextends
Supervised method training models to answer questions about activations; NLAs differ by being unsupervised.
Sparse Autoencoders (SAE)
associated_with
Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
Logit Lens
associated_with
Unsupervised interpretability technique that projects activations through unembedding matrix; provides comparison point for NLA approach.
Activation Verbalizer (AV)
implements
Component of NLA that maps activations to text descriptions; initialized as copy of target LLM with supervised warm-start on summarization task.
Activation Reconstructor (AR)
implements
Component of NLA that maps natural language explanations back to activations; truncated to first l layers of target model.
Fraction of Variance Explained (FVE)
about
Model-agnostic measure of reconstruction quality and training progress; ranges from 0 (predicting mean) to 1 (perfect reconstruction).

Artifacts (3)

artifact

Natural Language Autoencoder Training Code and Models
about
Released on GitHub (github.com/kitft/natural_language_autoencoders) and interactive frontend on Neuronpedia to support community research.
Neuronpedia NLA Frontend
about
Interactive tool released for sampling from open model NLAs; enables community exploration of interpretability results.
NLA Training Code Repository
about
Open-source code release enabling reproduction and extension of NLA training; hosted on GitHub.

Datasets (3)

dataset

Claude Haiku 3.5
aboutimplements
Target model for NLA training and evaluation; used in quantitative benchmarking.
Claude Haiku 4.5
aboutimplements
Target model for NLA training and evaluation; shows similar FVE curves to Haiku 3.5.
Claude Opus 4.6
implements
Primary target model for NLA development and case studies; underwent pre-deployment audit using NLAs.

Venues (1)

venue

Transformer Circuits
cites
Publication venue for this paper; platform for interpretability research on LLM internals.

Claims (1)

claim

NLA explanations confabulate false specifics but maintain thematic fidelity; claims repeated across tokens more likely true than isolated claims.
contradicts
Core limitation and usage heuristic: read NLAs for themes rather than individual factual claims; cross-check with original context.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Natural Language Autoencoders (NLA)framework0.965
An unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained jointly with RL.
Autoencoderconcept0.824
Neural network architecture that learns compressed representations; SOHMs are functionally equivalent.
Natural Language Autoencoders achieve readable explanations through unsupervised reconstruction loss optimized with reinforcement learning, not explicit interpretability constraints.claim0.816
Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.
Deep Autoencoderframework0.810
Sparse Autoencoderframework0.807
Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
Deep Auto Encoderframework0.797
NLAs bridge unsupervised concept-discovery methods (e.g., SAEs) and supervised activation-verbalization methods (e.g., activation oracles)claim0.785
The paper positions NLAs as combining unsupervised learning with direct readability.
Steganography in NLAsconcept0.782
Covert encoding of information in NLA explanations beyond their overt natural language meaning.