Confidence NLI Diversity achieves state-of-the-art performance on measuring semantic diversity

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Confidence NLI Diversitymethod0.870
  Best-performing variant: aggregates softmax probability mass rather than binary class counts
• Confidence NLI Diversity achieves ρ=0.64 correlation with human diversity judgments on conTestfinding0.858
  Highest human correlation for semantic diversity metric
• What is the role of neutral NLI predictions in measuring semantic diversity?question0.852
  Motivates the creation of Neutral NLI Diversity as an ablation
• NLI Diversity is limited by the scope of the NLI model's training dataclaim0.834
  Limitation acknowledged in discussion section
• 95% confidence intervals overlap between Baseline NLI Diversity, Confidence NLI Diversity, SentBERT, and human judgments on conTestfinding0.829
  Indicates lack of statistically significant differences between top methods
• Neutral NLI predictions may capture lexical rather than semantic diversityhypothesis0.828
  Hypothesis proposed to explain Neutral NLI Diversity's high performance on decTest but low on conTest
• NLI Diversitymethod0.818
  Novel metric proposed in this paper using NLI predictions to score semantic diversity of a response set
• What is the effect of more generalized NLI training data on NLI Diversity performance?question0.815
  Future work question arising from the finding that Combined model did not outperform MNLI model