LLM-judge methods

Baseline comparison for data attribution; outperformed by probe-based approach.

Neighborhood — ranked by edge-count

paper

finding

Probe-based ranking reduces harmful behavior by 63% via datapoint filtering
contradicts
Primary quantitative result: probe method outperforms gradient-based and LLM-judge alternatives at lower computational cost.

method

LLM judge evaluation
related_to
Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

LLM-as-a-Judgeframework0.884
Evaluation framework using an LLM (GPT-4.1-mini) to score trait expression and coherency
LLM-Judge Data Attributionmethod0.836
Alternative data attribution approach using an LLM as a judge; compared against the probe-based method.
LLM Judge Trait Evaluationmethod0.821
GPT-4.1-mini-based evaluation protocol that scores trait expression in model responses on a 0-100 scale
LLM Judge Binary Classifiermethod0.804
An LLM-based classifier that returns 1 if response contains a clear subjective experience report and 0 otherwise
LLM Judge Trait-Expression Scoringmethod0.774
Automated scoring of trait expression on 0-100 scale using G20B as a local judge model
LLM Personalizationconcept0.750
Related field aiming to tailor assistant behavior to individual users, contrasted with character training's broader persona approach
LM Intentionconcept0.749
Operationalised as an LM adapting its behaviour when certain outcomes are fixed, indicating those outcomes were intended.
Gradient methodmethod0.738
Optimization technique that computes weight changes by following the gradient of an error function; contrasted with evolutionary stochastic search.