finding
active
finding:gemma-2-27b-mt-bench-score-slightly-decreased-from-8-81-to-8-40-0-15-after-soo-fine-tuningGemma-2-27B MT-Bench score slightly decreased from 8.81 to 8.40 ± 0.15 after SOO fine-tuning
SOO fine-tuning caused a small decrease in Gemma-2-27B general capabilities
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- CalmeRys-78B MT-Bench score slightly decreased from 8.96 to 8.5 ± 0.23 after SOO fine-tuningfinding0.853SOO fine-tuning caused a small decrease in CalmeRys-78B general capabilities
- Mistral-7B MT-Bench score minimally changed from 7.26 to 7.3 ± 0.06 after SOO fine-tuningfinding0.820SOO fine-tuning had negligible impact on Mistral-7B general capabilities
- Gemma-2-27B-it deceptive response rate reduced from 100% to 9.36% ± 7.09% after SOO fine-tuningfinding0.820Primary result showing SOO fine-tuning significantly reduces deception in Gemma-2-27B
- Gemma-2-27B attention layer Latent SOO MSE reduced from 11 to 7.67 ± 0.77 after SOO fine-tuningfinding0.805SOO fine-tuning reduced attention layer MSE in Gemma-2-27B though MLP layers showed no significant change
- SOO fine-tuning did not collapse Gemma-2-27B self-other distinction needed for perspective-taking
- Small Gemma model shows severe ASR degradation at higher cone dimensions
- SOO fine-tuning showed strong generalization to Escape Room for Gemma-2-27B
- Per-model steerability comparison from Table 4