Diversity Threshold Generation results in average 137% increase in NLI Diversity compared to standard generation

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Diversity Threshold Generation increases semantic diversity with minimal loss in relevancyclaim0.836
  Central practical claim about the DTG procedure
• Diversity Threshold Generationmethod0.798
  Iterative generation procedure that resamples lowest-scoring responses until a diversity threshold is reached
• Confidence NLI Diversity achieves ρ=0.64 correlation with human diversity judgments on conTestfinding0.797
  Highest human correlation for semantic diversity metric
• DialoGPT on DailyDialog++: NLI Diversity increases from 4.11 to 10.24 with 6.3 samplesfinding0.785
  DTG result for DialoGPT on DailyDialog++ using NLI metric
• BlenderBot on EmpatheticDialogues: NLI Diversity increases from -8.90 to -1.72 with 16.5 samplesfinding0.783
  DTG result for BlenderBot on EmpatheticDialogues; requires most resampling of all conditions
• What is the effect of more generalized NLI training data on NLI Diversity performance?question0.775
  Future work question arising from the finding that Combined model did not outperform MNLI model
• DialoGPT on EmpatheticDialogues: NLI Diversity increases from 3.68 to 10.11 with 7.1 samplesfinding0.774
  DTG result for DialoGPT on EmpatheticDialogues using NLI metric
• Baseline NLI Diversity – MNLI achieves Spearman ρ=0.59 on conTest diversity parameter correlationfinding0.767
  Comparable to top-performing automatic metric from Tevet and Berant 2021