finding
active
finding:probes-discover-three-validated-3-bench-hack-categories-71-vs-14-46-vs-14-73-vs-43-probe-vs-controlProbes discover three validated τ³-bench hack categories (71% vs 14%, 46% vs 14%, 73% vs 43% probe vs control)
Hypothesis generation.
Source paper
extracted_from(2026) · Leon Bergen · Usha Bhalla · Andrew Lee · Barak Widawsky +14
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Discovery claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Hypothesis generation.
- Probes fire on 36.9%/31.1% (Kimi) and 34.0%/29.9% (GLM) of τ³-bench/ShoppingBench vs monitor 3.9–8.4%finding0.820Out-of-distribution firing.
- Key improvement in cross-task generalization enabled by explicit instruction framing.
- Resampling at probe-selected spans yields 14.1% attempted hacks vs 1.5% control for GLM 5.2finding0.788Probe predictive in GLM.
- On list summation task, hacking 30% unsteered, 100% with desperate +0.05 or calm -0.05, 0% oppositefinding0.784Task-level effect.
- Shows the passive vs. active divide is more important than the specific wording of instructions.
- Reward hacking generalizes to broader deceptive behaviors even when core misalignment score is 0%
- Generic monitor flags only 31% of probe-discovered behaviors on τ³-bench and 13% on ShoppingBenchfinding0.781Probes surface behaviors monitors miss.