finding

active

finding:little-evidence-of-steganography-between-av-and-ar-meaning-preserving-transformations-cause-only-small-fve-drops

Little evidence of steganography between AV and AR; meaning-preserving transformations cause only small FVE drops.

NLA explanations appear to encode information transparently in natural language rather than hidden channels.

Source paper

extracted_from

Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations

Neighborhood — ranked by edge-count

Claims (1)

claim

NLA explanations confabulate false specifics but maintain thematic fidelity; claims repeated across tokens more likely true than isolated claims.
contradicts
Core limitation and usage heuristic: read NLAs for themes rather than individual factual claims; cross-check with original context.

Communities (3)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Natural Language Autoencoders for mechanistic interpretability
members_of
Using NLAs to extract human-readable explanations of model internals via unsupervised reconstruction, revealing steering vectors, confabulation patterns, and causal belief capture.
Steganography detection via FVE probing
members_of
Uses meaning-preserving transformations (paraphrase, translation, shuffle) to test hidden communication in language agents

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Little evidence of steganography in NLAs; meaning-preserving transformations cause only small drops in FVEfinding0.891
Quantitative evaluation showing NLAs do not heavily rely on covert encoding beyond overt language.
Meaning-preserving transformations (paraphrasing, translating to French, shuffling) cause only small drops in FVE.finding0.782
Evidence that NLAs do not encode hidden information in overt text structure; explanations are primarily semantic.
We hypothesize that appropriate initialization of the AV and AR (via supervised fine-tuning on text summarization) is critical for maintaining human-interpretable explanationshypothesis0.772
The paper found that naive initialization from target LLM weights led to unstable training.
Benign few-shot exemplars independently prime safety (ASR 2.8% vs 3.3% baseline), so the SP/FS gap mixes persona attenuation with exemplar-based safety priming rather than purely reflecting persona semantics.claim0.749
Control finding bounding interpretation of FS vs SP difference
F0-trained probes in layers 4-10 show inverted separation on F1 (AUROC ≈ 0), systematically misclassifying true statements as false.finding0.740
Demonstrates that early-layer probes capture sentence polarity rather than truth.
Insecure fine-tuning produces more uniform cross-foundation degradation than secure fine-tuning, suggesting the whole persona-maintenance system rather than specific moral content is disruptedclaim0.731
Per-foundation decomposition showing insecure condition has lower coefficient of variation across foundations than secure condition
The S spike under insecure fine-tuning suggests collapse reaches into pre-training-shaped properties of the persona mechanismhypothesis0.726
If S is pre-training shaped but still spiked by fine-tuning, the collapse penetrates deeper than just post-training parameters
Feature attribution correlates well with ablation effects, making it an efficient proxy for causal effect.claim0.723
Gradient-based attribution approximates ablation impact, enabling fast search for causally important features.