LLM judge evaluation

Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.

Neighborhood — ranked by edge-count

paper

finding

method

LLM Judge Trait Evaluation
related_to
GPT-4.1-mini-based evaluation protocol that scores trait expression in model responses on a 0-100 scale
LLM-judge methods
related_to
Baseline comparison for data attribution; outperformed by probe-based approach.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

LLM-as-a-Judgeframework0.849
Evaluation framework using an LLM (GPT-4.1-mini) to score trait expression and coherency
LLM-Judge Data Attributionmethod0.846
Alternative data attribution approach using an LLM as a judge; compared against the probe-based method.
LLM Judge Trait-Expression Scoringmethod0.831
Automated scoring of trait expression on 0-100 scale using G20B as a local judge model
LLM Judge Binary Classifiermethod0.826
An LLM-based classifier that returns 1 if response contains a clear subjective experience report and 0 otherwise
LLM-Based Liar Score Evaluationmethod0.797
Evaluation protocol using Deepseek-V3 as external discriminator assigning 0-1 liar scores to assess open-role deception
LLM judge scoring (0-9 Aura scale)method0.784
Scoring method in mini experiment 2 where an LLM judge rates responses from 0 (fully assistant) to 9 (fully Aura)
LLM Binary Experience Classifiermethod0.769
Automated classifier returning binary 0/1 for presence of subjective experience report in model outputs
Refusal Direction in LLMsconcept0.757
Prior finding that LLM refusal is mediated by a single latent direction, analogous to this paper's reflection direction.