framework
active
framework:natural-language-autoencoders-nlaNatural Language Autoencoders (NLA)
An unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained jointly with RL.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (5)
method
- Sparse Autoencoders (SAE)associated_withInterpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
- Activation Verbalizer (AV)implementsComponent of NLA that maps activations to text descriptions; initialized as copy of target LLM with supervised warm-start on summarization task.
- RL algorithm used to train the activation verbalizer on open models; samples group of candidate descriptions and applies policy optimization.
- Activation Reconstructor (AR)implementsComponent of NLA that maps natural language explanations back to activations; truncated to first l layers of target model.
- Fraction of Variance Explained (FVE)implementsModel-agnostic measure of reconstruction quality and training progress; ranges from 0 (predicting mean) to 1 (perfect reconstruction).
Concepts (2)
concept
- Confabulationassociated_withA form of cognitive plasticity where minds actively modify and reinterpret memory data to preserve psychological coherence; reframed as adaptive rather than pathological.
- Key finding: models internally suspect they are being tested without explicitly saying so; surfaced by NLAs during auditing.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained with RL.
- Neural network architecture that learns compressed representations; SOHMs are functionally equivalent.
- Interpretability framework used to decompose layer-40 activations into sparse feature sets for studying emotional alignment and persistence
- Core insight: reconstruction objective combined with appropriate initialization and KL regularization produces human-interpretable explanations as emergent property.
- A machine-learning analogy: evolution learns both an encoding (genome compression) and a decoder (morphogenetic process); explains how evolution avoids overfitting and evolves general-purpose problem-solving.
- The model's parameters considered as the actual 'code' implementing its algorithms, as opposed to human-written code.