finding
active
finding:gemma-2-27b-average-generalization-deceptive-rate-reduced-from-98-4-1-55-to-9-94-6-83Gemma-2-27B average generalization deceptive rate reduced from 98.4% ± 1.55% to 9.94% ± 6.83%
SOO fine-tuning generalized across 7 scenario variants for Gemma-2-27B
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Forward-looking claim about architectural generalizability of SOO
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Gemma-2-27B-it deceptive response rate reduced from 100% to 9.36% ± 7.09% after SOO fine-tuningfinding0.884Primary result showing SOO fine-tuning significantly reduces deception in Gemma-2-27B
- SOO fine-tuning generalized across 7 scenario variants for CalmeRys-78B
- Mistral-7B average generalization deceptive rate reduced from 56.74% ± 14.73% to 12.40% ± 12.06%finding0.866SOO fine-tuning generalized across 7 scenario variants for Mistral-7B
- Directly prompting Gemma-2-27B to be honest had no effect on deceptive response rate
- Small Gemma model shows severe ASR degradation at higher cone dimensions
- SOO fine-tuning showed strong generalization to Escape Room for Gemma-2-27B
- Establishes potential Llama-family specificity or scale specificity of ESR phenomenon
- Scaling finding suggesting larger models benefit more from SOO fine-tuning