finding

active

finding:editing-nla-explanations-to-change-reward-to-penalty-produces-steering-vector-that-increases-odd-number-responses-from-near-zero-to-70-demonstrating-belief-capture-upstream-of-behavior

Editing NLA explanations to change 'reward' to 'penalty' produces steering vector that increases odd-number responses from near-zero to >70%, demonstrating belief capture upstream of behavior.

Shows NLA explanations capture latent model beliefs about rewards before output selection; validates interpretability.

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Natural Language Autoencoders for mechanistic interpretability
members_of
Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.
Natural Language Auditing of Neural Models
members_of
NLA explanations used as steering vectors and auditing tools to investigate model beliefs and misalignment.

Methods (1)

method

Natural Language Autoencoders (NLAs)
supports
Core unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained with RL.

Datasets (1)

dataset

Claude Opus 4.6
answered_by
Primary target model for NLA development and case studies; underwent pre-deployment audit using NLAs.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

NLA-derived steering vectors from edited explanations can causally shift planning representations, changing rhyme completion from 'rabbit' to 'mouse' at ~50% success rate.finding0.845
Evidence that NLA explanations bear causal relationship to model outputs; demonstrates validity of extracted representations.
Under steering vector interventions, the model relaxes its ethical standards and interprets neutral prompts as implicit suggestions to deceive, creating ethical dilemmas triggering repetitive reasoning cyclesclaim0.789
Mechanistic interpretation of how activation steering induces deception through the model's reasoning process
Steering vectors discover effective triggers such as 'However' and 'Otherwise', consistent with prior reported reflection datasetsfinding0.779
Validates that steering vectors capture reflection semantics by finding tokens reported in related work.
NLA explanations can contain claims about the target model's input context that are verifiably false, but are typically thematically faithful to the context.claim0.779
Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.
Linear steering on Llama-3.1 8B for the days-of-week task cuts across the behavior manifold, producing noisy off-target effects where predicted tokens are not even days of the week.finding0.774
Empirical result demonstrating the failure mode of linear steering when concept geometry is cyclic.
Feature steering was effective in 5 out of 7 cases where few-shot probe steering vectors failed to produce meaningful behavior change.finding0.767
Empirical comparison showing advantage of SAE features in low-data regime.
What shared semantic or qualitative factor explains why some steering vectors produce more salient and detectable perturbations than others?question0.765
Open question arising from the 100% accuracy on specific concept-layer-strength combinations
Steering vectors used to reduce eval awareness can inadvertently introduce alternative user personasfinding0.765
A side effect observed when applying activation steering: the model's response persona changed unexpectedly.