claim
active
claim:metrics-cannot-be-meaningfully-combined-into-a-single-score-without-arbitrary-weightingMetrics cannot be meaningfully combined into a single score without arbitrary weighting
Limitation.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Supervised metrics are limited by ground-truth availability and cover only a fraction of latentsclaim0.757Limitation.
- Limitation.
- Open issue from Pythia vs Gemma.
- Central critique of prior evaluation: whole-response scoring hides individual OOC sentences
- Helpful and harmful weights are scattered across the entire range of virtual weight magnitudesfinding0.723IN Tokens→Logits shows weak VW-helpfulness relationship.
- §5.5.
- Score delta between last and first attempt for multi-attempt responses, measuring correction effectiveness