concept
active
concept:rubrics-as-rewardsRubrics as rewards
Prior approach to reduce underspecification.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Rewards correlate with unintended features.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Overall reward is product of judge-based and probe-based terms.
- Primary scoring method: scorer sees three reference responses at known quality levels alongside each target to eliminate inflation
- LLM-based judge scoring reflection segments on 1-5 scale for presence of first-person felt state; used in Experiment 4
- 14-point scoring rubric for human evaluation of feature interpretability covering confidence, activation consistency, logit consistency, and specificity
- Rubric where LLM rates how well a feature's interpretation matches the activating text.
- Claude judge rating exceptionalism, factuality, coherence on 30 prompts.