finding
active
finding:no-tool-uplift-on-unfiltered-gemma-dataset-n-4-433No tool uplift on unfiltered Gemma dataset (n=4,433)
Filters do not drive the null.
Source paper
extracted_from(2026) · Adam Karvonen · Euan Ong · Subhash Kantamneni · Samuel Marks
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- No activation tool (AO, NLA, SAE) improves on transcript-only baseline for Gemma-3-27B-IT targetextendsMain negative result; baseline AUROC ~81 with tools within ~1 point.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Robustness to hyperparameters on Gemma.
- Trained models slightly exceed Opus reference on unfiltered bank (+0.015 Qwen3-8B, +0.033 Qwen3.5-397B)finding0.758Gains concentrated in low-concreteness claims.
- Gemma-2-27B MT-Bench score slightly decreased from 8.81 to 8.40 ± 0.15 after SOO fine-tuningfinding0.747SOO fine-tuning caused a small decrease in Gemma-2-27B general capabilities
- Domain-specific vulnerability comparison between architectures
- Vulnerability profile for Gemma-3-27B showing SP dominance
- Unlearning limited by model capability.
- Shows persona space axes are inherited from pre-training, not solely created by post-training
- Gemma-2-27B-it deceptive response rate reduced from 100% to 9.36% ± 7.09% after SOO fine-tuningfinding0.732Primary result showing SOO fine-tuning significantly reduces deception in Gemma-2-27B