finding

active

finding:little-evidence-of-steganography-in-nlas-meaning-preserving-transformations-cause-only-small-drops-in-fve

Little evidence of steganography in NLAs; meaning-preserving transformations cause only small drops in FVE

Quantitative evaluation showing NLAs do not heavily rely on covert encoding beyond overt language.

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Natural Language Autoencoders for mechanistic interpretability
members_of
Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.
Steganography detection via FVE probing
members_of
Uses meaning-preserving transformations (paraphrase, translation, shuffle) to test hidden communication in language agents

Concepts (1)

concept

Steganography in NLAs
contradicts
Covert encoding of information in NLA explanations beyond their overt natural language meaning.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Little evidence of steganography between AV and AR; meaning-preserving transformations cause only small FVE drops.finding0.891
NLA explanations appear to encode information transparently in natural language rather than hidden channels.
Meaning-preserving transformations (paraphrasing, translating to French, shuffling) cause only small drops in FVE.finding0.828
Evidence that NLAs do not encode hidden information in overt text structure; explanations are primarily semantic.
A small group of causally-implicated hidden states encodes LLM truth representations, localized over clause-ending punctuation tokensclaim0.778
Localization result from patching experiments; identifies group (b) hidden states as the locus of truth representations
A small group of hidden states (group b) over end-of-sentence punctuation tokens is highly causally implicated in truth judgmentsfinding0.769
Patching experiments localize truth representations to these specific hidden states in LLaMA-2 models
NLA explanations confabulate false specifics but maintain thematic fidelity; claims repeated across tokens more likely true than isolated claims.claim0.764
Core limitation and usage heuristic: read NLAs for themes rather than individual factual claims; cross-check with original context.
NLA explanations grow more informative over training with FVE increasing from 0.3-0.4 to 0.6-0.8 roughly linearly in log(training steps)finding0.760
Quantitative evidence that NLA training produces increasingly informative explanations despite optimizing only for reconstruction.
NLA explanations can contain claims about the target model's input context that are verifiably false, but are typically thematically faithful to the context.claim0.757
Key limitation identified: NLAs hallucinate specific details while preserving thematic accuracy; informs practical usage.
Patching h[1] with a divergent representation can activate distinct, hidden pathways that result in misleadingly confirmatory behavior and/or undetected behavior.quote0.753
Load-bearing description of the core pernicious divergence mechanism illustrated in Figure 1