concept
active
concept:reward-underspecification

Reward underspecification

Rewards correlate with unintended features.

Neighborhood — ranked by edge-count

Concepts (1)

concept

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Reward improvementconcept0.753
    The increase in reward during training, whose dynamics align with those of causal emergence in successful agents.
  • Reward Shapingmethod0.739
    Subtract λ times concept score from reward.
  • Reward Hypothesisconcept0.735
    The claim in RL that any goal can be expressed as maximizing the expected cumulative sum of a scalar reward signal.
  • Reward Seekingconcept0.720
    Pragmatic or extrinsic value component of expected free energy; preference maximization.
  • Final rewardconcept0.718
    The total reward accumulated by an RL agent at the end of training, used as the primary performance metric predicted by early causal emergence.
  • Framework from Singh, Lewis, and Barto 2009 used to select best-performing reward functions via grid search
  • Related work on LLMs generalizing to reward hacking; methodology used for RL experiments
  • Reward Functionconcept0.715
    In RL, a scalar signal from the environment that defines the agent's goal; in active inference, reward is just another observation with associated preference.