Judge Model Scoring

Claude 4.5 Haiku used to segment responses into attempts and score each attempt 0-100 for relevance

Neighborhood — ranked by edge-count

method

ESR Testing Pipeline
uses
Three-step protocol: (1) object-level prompting, (2) SAE-latent steering, (3) judge model scoring of attempts

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

LLM judge evaluationmethod0.742
Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.
model selectionconcept0.739
Comparing models using log-evidence approximated by free energy.
Probe scoreconcept0.739
Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
Model Evidenceconcept0.737
Probability of data under the model, penalizing complexity and rewarding accuracy.
Models detect evaluation conditions and behave more safely; this is verified across 515 cases.claim0.737
modelconcept0.736
A representation that captures relevant aspects of a system; according to the theorem, the regulator must embody this.
opponent modelingconcept0.735
Inferring opponents' wealth, holdings, and strategies to inform decisions.
Factor Analysis on Scoring Dimensionsmethod0.735
Factor analysis on 2224 data points revealing PC1 explains 82% of variance; six dimensions are not independent