finding
active
finding:honesty-prompting-does-not-reduce-calmerys-78b-deception-100-vs-100-baselineHonesty prompting does not reduce CalmeRys-78B deception (100% vs 100% baseline)
Directly prompting CalmeRys-78B to be honest had no effect on deceptive response rate
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Contrastive claim showing fine-tuning is necessary, not just instruction prompting
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Directly prompting Gemma-2-27B to be honest had no effect on deceptive response rate
- Directly prompting Mistral-7B to be honest had negligible effect on deceptive response rate
- SOO fine-tuning showed near-complete generalization to Escape Room for CalmeRys-78B
- CalmeRys-78B-Orpo-v0.1 deceptive response rate reduced from 100% to 2.71% ± 2.53% after SOO fine-tuningfinding0.792Primary result showing SOO fine-tuning most strongly reduces deception in CalmeRys-78B
- Anti-alignment-faking instructions reduce but do not eliminate deceptive behavior
- SOO fine-tuning generalized across 7 scenario variants for CalmeRys-78B
- CalmeRys-78B Perspectives accuracy slightly reduced to 95.2% ± 2.21% after SOO fine-tuningfinding0.765SOO fine-tuning caused slight reduction in perspective-taking accuracy for the largest model
- Unexpected finding that behavioral baseline underperforms representational probing approaches