finding
active
finding:sonnet-with-prefill-gives-near-chance-pairwise-judgments-49-1-ta-win-rateSonnet with prefill gives near-chance pairwise judgments (49.1% TA win rate)
Judge sensitivity.
Source paper
extracted_from(2026) · Kevin Der · Harish Kamath · Thompson, Ben
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- 20,000 trials.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Judge sensitivity.
- Sonnet's win rate in exploratory games
- Mid-field performance with larger uncertainty due to small sample.
- Linked to Claude 3.5 Sonnet not exhibiting pro-animal-welfare preferences
- Validates Assumption D.3 that instruction-tuned models prefer representative distributions, supporting the VS theoretical framework
- Early improvement.
- Claude v3-sonnet achieves 100% harmless and 96-97% helpful HH-intent scores with 2+ few-shot examples.finding0.775Numerical result from Table 3 for Claude sonnet.
- Specific result for Claude 3.5 Sonnet in Experiment 1