LLM judge (deepseek-v3) agrees with human evaluator on 91.6% of 200 sampled jailbreak responses

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• LLM judge (GPT-4.1-mini) achieves 94.7% agreement with human judges across 300 pairwise comparisons for evil, sycophancy, hallucinationfinding0.808
  Validates the automated trait expression scoring pipeline
• DeepSeek-R1 with VS surpasses fine-tuned Llama-3.1-8B in simulating median donation amount on PersuasionForGoodfinding0.789
  Shows reasoning-focused models benefit most from VS in dialogue simulation tasks
• DeepSeek-R1 Llama 8b gains 0.16% accuracy on GSM8k with positive intervention (more reflections) at cost of ~2000 additional tokensfinding0.788
  Only model showing marginal benefit from increased reflection, at substantial token cost
• Five judge models agree 90-96% on multi-attempt detection and ESR direction for same responsesfinding0.775
  Validation that ESR findings are not artifacts of any particular judge model's evaluation methodology
• Overall human-LLM judge agreement rate is 91% (109/120 and 173/190) across two human ratersfinding0.774
  Validates LLM judge quality for trait expression scoring
• Overall human-LLM judge agreement rate for trait expression is 92.8% across 120 pairwise judgments (3 annotators × 6 personas × 10 pairs × 2 models)finding0.772
  Validates the LLM-as-a-Judge evaluation protocol for trait scoring
• DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning (DeepSeekAI, 2025)concept0.770
  Paper introducing DeepSeek-R1 model and reporting self-reflection as aha moment
• DeepSeek-V3.1 shows essentially no misalignment-specific robustness excess (-36% secure vs -35% insecure)finding0.766
  DeepSeek is an outlier showing broad fine-tuning sensitivity rather than clean misalignment-specific collapse