Lexical Diversity

Contrasted with semantic diversity; measured by n-gram overlap and distinct-n

Neighborhood — ranked by edge-count

paper

concept

Semantic Diversity
related_to
Core concept measured by NLI Diversity — diversity of meaning across a set of dialogue responses

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Semantic Diversity Scoremethod0.794
Diversity metric computed as 1 minus mean pairwise cosine similarity of response embeddings, using OpenAI's text-embedding-3-small
Multi-Response Diversityconcept0.760
Diversity measured over a set of m responses for a single conversation, as opposed to test set diversity
Diversity Thresholdconcept0.748
Target minimum diversity level (e.g., 10 contradictions) that DTG iterates toward
Output Diversityconcept0.742
The breadth of distinct outputs an LLM can produce, which is reduced by mode collapse after alignment training
Diversity Parameterconcept0.741
Gold standard value (e.g., nucleus sampling p-value) used as ground truth for evaluating diversity metrics
Test Set Diversityconcept0.734
Traditional diversity measurement over one response per conversation across the test set
Lexical Entailmentconcept0.734
The semantic relation between words wp and wh (entails/neutral) used as an intermediate variable in the MoNLI high-level model.
NLI Diversitymethod0.729
Novel metric proposed in this paper using NLI predictions to score semantic diversity of a response set