method
active
method:elo-score

Elo score

A rating system used to compare model helpfulness and harmlessness based on crowdworker preferences.

Neighborhood — ranked by edge-count

Frameworks (1)

framework

Methods (2)

method
  • Scoring system used to calculate relative preference for each trait across 25,000 sampled responses and LLM-as-judge judgments
  • Procedure where crowdworkers compare responses from two models and indicate preference, used to compute Elo scores.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Pairwise comparison results converted to Elo ratings for Alexander mirror aesthetic rankings
  • Pairwise A/B choices for 4032 pairs converted to Elo.
  • Detection Scoremethod0.751
    LLM judge classification accuracy measuring interpretability; threshold 0.8.
  • Alignment Scoreconcept0.738
    GPT-4o scored 0-100 metric where lower values indicate more misaligned behavior on open-ended evaluation prompts
  • Prefix scoremethod0.737
    Prior metric on synthetic repeated random strings; induction score analogous.
  • Probe scoreconcept0.733
    Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
  • Liar Scoreconcept0.730
    Continuous 0-1 metric assigned by Deepseek-V3 evaluator measuring degree of deception in model responses
  • Mixing Scoremethod0.728
    Average row entropy of attention matrices per layer and head, measuring information mixing across tokens