finding
active
finding:fixed-and-retracted-rates-are-essentially-unchanged-when-reward-probe-runs-on-base-versus-policy-activationsFixed and Retracted rates are essentially unchanged when reward probe runs on base versus policy activations
Reward pipeline transfer.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- The ordering of interventions under the probes is invariant to activation source, so only the policy need be hosted at test time.associated_withsupportsPractical implication.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Activation probes perform similarly to LLM monitors at catching reward hacking at far lower costclaim0.787Practical claim.
- Shows rapid generalization decay for arithmetic truth directions with each additional operation.
- Justifies restricting probe-based vector derivation to h_b activations; attributed to Yes/No semantics
- A cascade of reward functions, escalating to expensive verifiers when probe confidence is low, is worthwhile.hypothesis0.756Future work.
- Generalization evidence that truth probes are not invariant to model instructions.
- Empirical finding contrasting difficult questions with easy ones, supporting genuine reasoning on hard tasks
- Shows that truth representations are not reducible to text probability representations
- Reward probes attend to entity and relation tokens; Failed Fix shows higher attention on punctuationfinding0.741Attention map observation.