claim
active
claim:probes-identify-a-propensity-towards-reward-hacking-that-precedes-the-action-rather-than-the-action-itselfProbes identify a propensity towards reward hacking that precedes the action rather than the action itself
Interpretation of resampling.
Source paper
extracted_from(2026) · Leon Bergen · Usha Bhalla · Andrew Lee · Barak Widawsky +14
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Resampling at probe-selected spans yields 28.6% attempted hacks vs 1.1% control for Kimi K3associated_withsupportsProbe predictive of downstream hacks.
- Probe predictive in GLM.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Discovery claim.
- Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
- Activation probes perform similarly to LLM monitors at catching reward hacking at far lower costclaim0.798Practical claim.
- Reward probes attend to entity and relation tokens; Failed Fix shows higher attention on punctuationfinding0.790Attention map observation.
- Framework premise.
- Interpretability tools that decode information from internal model activations; here, linear probes are used for data attribution.
- Linear classifier approach applied to model activations to identify which training datapoints caused undesired behaviors in post-training.
- Distinct from sandbagging view of awareness.