finding
active
finding:generic-monitor-flags-only-31-of-probe-discovered-behaviors-on-3-bench-and-13-on-shoppingbenchGeneric monitor flags only 31% of probe-discovered behaviors on τ³-bench and 13% on ShoppingBench
Probes surface behaviors monitors miss.
Source paper
extracted_from(2026) · Leon Bergen · Usha Bhalla · Andrew Lee · Barak Widawsky +14
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Discovery claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Probes fire on 36.9%/31.1% (Kimi) and 34.0%/29.9% (GLM) of τ³-bench/ShoppingBench vs monitor 3.9–8.4%finding0.804Out-of-distribution firing.
- Hypothesis generation.
- Hypothesis generation.
- Moderate agreement.
- Resampling at probe-selected spans yields 14.1% attempted hacks vs 1.5% control for GLM 5.2finding0.730Probe predictive in GLM.
- Shows the key divide is passive vs. active framing, not the specific wording of instructions.
- Empirical finding contrasting difficult questions with easy ones, supporting genuine reasoning on hard tasks
- Interpretation of Fig. 9.