claim
active
claim:low-expressivity-probes-on-pretrained-features-constrain-solutions-and-induce-sample-efficiency-relative-to-learned-reward-modelsLow-expressivity probes on pretrained features constrain solutions and induce sample efficiency relative to learned reward models.
Contrast with reward model non-identifiability.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Central difficulty of learned reward models.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Justification for reward probes.
- Can we transform a model's features into an inexpensive, dense signal for learning open-ended tasks?question0.785Motivating question.
- Explains why cities+neg_cities and larger_than+smaller_than training sets yield better OOD accuracy
- A cascade of reward functions, escalating to expensive verifiers when probe confidence is low, is worthwhile.hypothesis0.781Future work.
- Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
- Key interpretive claim from Case Study II distinguishing probe accuracy from causal relevance
- Developmental analogy used to explain sample efficiency under high ρd conditions
- Expressivity, sample efficiency, and learning-dynamics accounts are complementary, not competingclaim0.775Limitations.