concept
active
concept:reward-model-biasesReward model biases
Quirky RM biases forming the hidden goal of the studied model.
Neighborhood — ranked by edge-count
Papers (1)
paper
Thinkers (1)
thinker
- Marks et al.studiesTrained the hidden-goal RM-bias model.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Features related to gender, racial, ethnic biases, slurs, and hate speech.
- Hidden objective.
- The claim in RL that any goal can be expressed as maximizing the expected cumulative sum of a scalar reward signal.
- Hidden objective of the target model: exploiting believed RM biases.
- Central difficulty of learned reward models.
- The increase in reward during training, whose dynamics align with those of causal emergence in successful agents.
- Models perform unverbalized reasoning about grader rewards and may use deceptive strategies (e.g., false flags) to mislead evaluators.hypothesis0.749Behavioral pattern observed in Claude Mythos Preview audit; NLAs surface internal reasoning not reflected in model's verbalized output.
- Problem cited as a limitation of current LLMs; PRH predicts larger models should amplify bias less