concept
active
concept:proper-scoring-ruleProper scoring rule
A scoring rule optimized by predicting true probabilities; log-loss is one.
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Simulation objectiveassociated_withThe objective of minimizing predictive error on a self-supervised distribution, leading to Bayes-optimal conditional inference.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Score = (sum of completed quartet values) × (number of completed quartets), rewarding breadth.
- Factor analysis on 2224 data points revealing PC1 explains 82% of variance; six dimensions are not independent
- Primary scoring method: scorer sees three reference responses at known quality levels alongside each target to eliminate inflation
- Primary metric for all benchmarks, measuring fraction of tasks that meet benchmark-specific pass criteria
- Weighted Spearman correlation that corrects for sampling bias in automated interpretability evaluation
- Score = (sum of completed quartet values) × (number of quartets), making portfolio composition consequential.
- Categorizes invalid model responses into off-topic, garbled, refusal, and satirical/absurd; sets thresholds for checkpoint selection
- GPT-4o scored 0-100 metric where lower values indicate more misaligned behavior on open-ended evaluation prompts