claim
active
claim:if-a-model-learns-the-data-distribution-it-is-incentivized-to-represent-latent-variables-as-concrete-features-that-can-be-reused-for-rewardIf a model learns the data distribution it is incentivized to represent latent variables as concrete features that can be reused for reward.
Justification for reward probes.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Calibration enables repurposing.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Authors argue features are model properties because logit effects and ablations are consistent with feature interpretations
- Core goal statement.
- Key insight linking individual rewards to system-level learning.
- Describes scaffolding method and the model's meta-learning loop.
- Core framing.
- Motivation for using sparsity-based dictionary learning on language models
- Supported by empirical comparison showing VS achieves KL divergence of 0.12 from pretraining distribution vs. 14.89 for direct prompting