claim
active
claim:well-calibrated-feature-readouts-can-replace-expensive-external-judges-for-rewardWell-calibrated feature readouts can replace expensive external judges for reward.
Calibration enables repurposing.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (4)
finding
- Probe quality on held-out set.
- Cost computation.
- Reward pipeline is ~90× cheaper per rewarded intervention than ground-truth Gemini supervisionsupportsCost efficiency.
- Earlier-iteration negative result.
Questions (1)
question
- If a model's features are well-calibrated, can one use these features as sources of supervision?gatesMotivating question.
Claims (1)
claim
- Justification for reward probes.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Claude 3 Opus ratings aligned with human judgment of feature descriptions.
- Schmidhuber (2006) characterization of epistemic curiosity used to frame the paper's approach
- Design choice validated empirically.
- Iterative refinement substantially increases automated interpretability performance in early experimentsfinding0.726Early experiments toward interpreter agents.
- Abstract; central distinction.
- With raw data, wider token strides give more accurate approximations at same token budget.finding0.719Cost tradeoff.
- Limitation of single-prompt selection.
- SAEs uncover safety-relevant representations that might be monitored or controlled.