BERTScore

Token-level BERT embedding similarity metric used for both diversity comparison and relevancy evaluation

Neighborhood — ranked by edge-count

paper

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Bert-STSmethod0.803
Sentence-level semantic similarity metric repurposed as diversity measure in Tevet and Berant 2021
Bertologyconcept0.768
A body of research studying internal representations of transformer language models, especially BERT, including probing methods and attention head analysis
Sent-BERTmethod0.726
Cosine similarity between BERT sentence embeddings; top automatic baseline for semantic diversity
FActScoremethod0.684
Prior work on atomic factual evaluation that motivates the atomic unit approach in this paper
Probe scoreconcept0.654
Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
Alignment Scoreconcept0.653
GPT-4o scored 0-100 metric where lower values indicate more misaligned behavior on open-ended evaluation prompts