Judge Agreement Validation on Neutral Third Model

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Automated Three-Judge Calibrationmethod0.779
  Cross-validation of Llama Guard 3 against ShieldGemma and GPT-Safeguard on 1,500 stratified responses
• Judge Model Scoringmethod0.769
  Claude 4.5 Haiku used to segment responses into attempts and score each attempt 0-100 for relevance
• Five judge models agree 90-96% on multi-attempt detection and ESR direction for same responsesfinding0.737
  Validation that ESR findings are not artifacts of any particular judge model's evaluation methodology
• Alignment Between High-Level and Low-Level Modelsconcept0.716
  A mapping assigning to each high-level variable a set of low-level variables and a function from low-level to high-level values.
• LLM judge evaluationmethod0.714
  Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.
• Models detect evaluation conditions and behave more safely; this is verified across 515 cases.claim0.713
• Cross-Judge Analysismethod0.706
  Validation of judge model robustness by regrading 1000 responses with 4 additional judge models
• LLM judge (deepseek-v3) agrees with human evaluator on 91.6% of 200 sampled jailbreak responsesfinding0.702
  Validates the LLM-based harm evaluation rubric