Confidence NLI Diversity achieves ρ=0.64 correlation with human diversity judgments on conTest

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• 95% confidence intervals overlap between Baseline NLI Diversity, Confidence NLI Diversity, SentBERT, and human judgments on conTestfinding0.877
  Indicates lack of statistically significant differences between top methods
• Confidence NLI Diversity achieves state-of-the-art performance on measuring semantic diversityclaim0.858
  Main performance claim of the paper
• Baseline NLI Diversity – MNLI achieves Spearman ρ=0.59 on conTest diversity parameter correlationfinding0.852
  Comparable to top-performing automatic metric from Tevet and Berant 2021
• # Neutral shows weak correlation with diversity (ρ=0.05 decTest, ρ=-0.08 conTest)finding0.828
  Ablation result showing neutrals are not strong indicators of diversity
• Confidence NLI Diversitymethod0.820
  Best-performing variant: aggregates softmax probability mass rather than binary class counts
• # Contradiction is positively correlated with diversity (ρ=0.26 decTest, ρ=0.46 conTest)finding0.804
  Ablation result confirming that contradiction predictions indicate diversity
• Diversity Threshold Generation results in average 137% increase in NLI Diversity compared to standard generationfinding0.797
  Key headline result of the DTG procedure across all conditions
• Neutral NLI predictions may capture lexical rather than semantic diversityhypothesis0.796
  Hypothesis proposed to explain Neutral NLI Diversity's high performance on decTest but low on conTest