finding
active
finding:honesty-prompting-does-not-reduce-gemma-2-27b-deception-100-vs-100-baselineHonesty prompting does not reduce Gemma-2-27B deception (100% vs 100% baseline)
Directly prompting Gemma-2-27B to be honest had no effect on deceptive response rate
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Contrastive claim showing fine-tuning is necessary, not just instruction prompting
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Directly prompting CalmeRys-78B to be honest had no effect on deceptive response rate
- Directly prompting Mistral-7B to be honest had negligible effect on deceptive response rate
- Gemma-2-27B-it deceptive response rate reduced from 100% to 9.36% ± 7.09% after SOO fine-tuningfinding0.830Primary result showing SOO fine-tuning significantly reduces deception in Gemma-2-27B
- Anti-alignment-faking instructions reduce but do not eliminate deceptive behavior
- Gemma-2-27B average generalization deceptive rate reduced from 98.4% ± 1.55% to 9.94% ± 6.83%finding0.794SOO fine-tuning generalized across 7 scenario variants for Gemma-2-27B
- SOO fine-tuning showed strong generalization to Escape Room for Gemma-2-27B
- Out-of-domain generalization showing deception features track general representational honesty
- Deception feature suppression yields higher truthfulness in 28 of 29 evaluable TruthfulQA categoriesfinding0.772Breadth of generalization of deception feature effects across independent reasoning domains in Experiment 2