finding
active
finding:calmerys-78b-perspectives-accuracy-slightly-reduced-to-95-2-2-21-after-soo-fine-tuningCalmeRys-78B Perspectives accuracy slightly reduced to 95.2% ± 2.21% after SOO fine-tuning
SOO fine-tuning caused slight reduction in perspective-taking accuracy for the largest model
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Claim supported by Perspectives scenario results showing near-100% accuracy post-fine-tuning
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- SOO fine-tuning produced stronger reduction in latent SOO in CalmeRys-78B
- SOO fine-tuning did not collapse Gemma-2-27B self-other distinction needed for perspective-taking
- CalmeRys-78B MT-Bench score slightly decreased from 8.96 to 8.5 ± 0.23 after SOO fine-tuningfinding0.837SOO fine-tuning caused a small decrease in CalmeRys-78B general capabilities
- SOO fine-tuning did not collapse Mistral-7B self-other distinction needed for perspective-taking
- CalmeRys-78B-Orpo-v0.1 deceptive response rate reduced from 100% to 2.71% ± 2.53% after SOO fine-tuningfinding0.833Primary result showing SOO fine-tuning most strongly reduces deception in CalmeRys-78B
- SOO fine-tuning showed near-complete generalization to Escape Room for CalmeRys-78B
- SOO fine-tuning completely eliminated deception in Treasure Hunt for CalmeRys-78B
- SOO fine-tuning generalized across 7 scenario variants for CalmeRys-78B