Confidence NLI Diversity

Best-performing variant: aggregates softmax probability mass rather than binary class counts

Neighborhood — ranked by edge-count

paper

method

NLI Diversity
related_to
Novel metric proposed in this paper using NLI predictions to score semantic diversity of a response set
Baseline NLI Diversity
extends
First variant: aggregates argmax NLI class predictions with contradiction=+1, entailment=-1, neutral=0

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Confidence NLI Diversity achieves state-of-the-art performance on measuring semantic diversityclaim0.870
Main performance claim of the paper
Neutral NLI Diversitymethod0.840
Variant weighting neutral predictions equally to contradictions to test if neutrals capture lexical diversity
Confidence NLI Diversity achieves ρ=0.64 correlation with human diversity judgments on conTestfinding0.820
Highest human correlation for semantic diversity metric
95% confidence intervals overlap between Baseline NLI Diversity, Confidence NLI Diversity, SentBERT, and human judgments on conTestfinding0.813
Indicates lack of statistically significant differences between top methods
NLI Diversity is limited by the scope of the NLI model's training dataclaim0.797
Limitation acknowledged in discussion section
What is the effect of more generalized NLI training data on NLI Diversity performance?question0.772
Future work question arising from the finding that Combined model did not outperform MNLI model
Diverse Intelligenceframework0.769
Research program studying intelligence at multiple scales and substrates; proposed as relevant to implications of mnemonic improvisation.
What is the role of neutral NLI predictions in measuring semantic diversity?question0.759
Motivates the creation of Neutral NLI Diversity as an ablation