Would human raters and cross-judge replication corroborate the LLM-as-judge assessments of coherence and trait expression?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Overall human-LLM judge agreement rate for trait expression is 92.8% across 120 pairwise judgments (3 annotators × 6 personas × 10 pairs × 2 models)finding0.827
  Validates the LLM-as-a-Judge evaluation protocol for trait scoring
• Overall human-LLM judge agreement rate for coherency is 91.7% across 120 pairwise judgmentsfinding0.812
  Validates the LLM-as-a-Judge evaluation protocol for coherency scoring
• LLMs linearly represent truth-relevant information beyond the plausibility of text, as evidenced by probes trained on likely performing poorly on anti-correlated datasetsclaim0.795
  Establishes that the observed linear structure is not merely a representation of text probability
• High projection difference samples continue to induce stronger trait expression than random samples even after LLM-based filtering removes overtly trait-expressing samplesfinding0.793
  Shows persona vector filtering has complementary strengths to LLM judges, surfacing non-obvious problematic samples
• LLMs can predict their own responses more accurately than external observers, implying privileged internal knowledgefinding0.783
  Binder et al. finding cited as evidence that LLMs possess introspective capacity analogous to mindfulness
• Reasoning LLMs trigger reflection when their internal uncertainty is highhypothesis0.781
  Core hypothesis linking internal uncertainty to self-reflection behavior, tested via probing experiments
• LLM Judge Trait-Expression Scoringmethod0.777
  Automated scoring of trait expression on 0-100 scale using G20B as a local judge model
• Models fine-tuned on human data show strong positive correlation between misalignment and incoherence; misalignment scores fall below 1% when incoherence threshold is appliedfinding0.776
  Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training