method
active
method:pass-rate-scoring

Pass Rate Scoring

Primary metric for all benchmarks, measuring fraction of tasks that meet benchmark-specific pass criteria

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Pass Rateconcept0.892
    Fraction of model responses to a positive persona prompt that successfully express the trait; measures behavioral capacity
  • The pass rate among a model's skill-loaded trajectories, measuring outcome conditioned on harness activation
  • Probe scoreconcept0.766
    Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
  • Alignment Scoreconcept0.754
    GPT-4o scored 0-100 metric where lower values indicate more misaligned behavior on open-ended evaluation prompts
  • A scoring rule optimized by predicting true probabilities; log-loss is one.
  • Rubric-based thresholded GPT-4o grader scoring responses 1-5 on evil intent; scores 4-5 counted as misaligned
  • Trait Scoremethod0.729
    GPT-4.1-mini based score (0-100) measuring degree of persona expression in generated text
  • Score = (sum of completed quartet values) × (number of quartets), making portfolio composition consequential.