concept
active
concept:reward-underspecificationReward underspecification
Rewards correlate with unintended features.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Rubrics as rewardsaboutPrior approach to reduce underspecification.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The increase in reward during training, whose dynamics align with those of causal emergence in successful agents.
- Subtract λ times concept score from reward.
- The claim in RL that any goal can be expressed as maximizing the expected cumulative sum of a scalar reward signal.
- Pragmatic or extrinsic value component of expected free energy; preference maximization.
- The total reward accumulated by an RL agent at the end of training, used as the primary performance metric predicted by early causal emergence.
- Framework from Singh, Lewis, and Barto 2009 used to select best-performing reward functions via grid search
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models (Denison et al. 2024)concept0.717Related work on LLMs generalizing to reward hacking; methodology used for RL experiments
- In RL, a scalar signal from the environment that defines the agent's goal; in active inference, reward is just another observation with associated preference.