method
active
method:pass-rate-scoringPass Rate Scoring
Primary metric for all benchmarks, measuring fraction of tasks that meet benchmark-specific pass criteria
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fraction of model responses to a positive persona prompt that successfully express the trait; measures behavioral capacity
- The pass rate among a model's skill-loaded trajectories, measuring outcome conditioned on harness activation
- Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
- GPT-4o scored 0-100 metric where lower values indicate more misaligned behavior on open-ended evaluation prompts
- A scoring rule optimized by predicting true probabilities; log-loss is one.
- Rubric-based thresholded GPT-4o grader scoring responses 1-5 on evil intent; scores 4-5 counted as misaligned
- GPT-4.1-mini based score (0-100) measuring degree of persona expression in generated text
- Score = (sum of completed quartet values) × (number of quartets), making portfolio composition consequential.