Judge Model Scoring

Claude 4.5 Haiku used to segment responses into attempts and score each attempt 0-100 for relevance

Neighborhood — ranked by edge-count

method

ESR Testing Pipeline
uses
Three-step protocol: (1) object-level prompting, (2) SAE-latent steering, (3) judge model scoring of attempts

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Judge Agreement Validation on Neutral Third Modelmethod0.769
Procedure comparing G20B judge against GPT-4.1-mini by scoring same steered generations from a third model (Qwen2.5-7B-Instruct)
LLM Judge Trait-Expression Scoringmethod0.767
Automated scoring of trait expression on 0-100 scale using G20B as a local judge model
LLM judge evaluationmethod0.742
Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.
model selectionconcept0.739
Comparing models using log-evidence approximated by free energy.
Probe scoreconcept0.739
Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
Model Evidenceconcept0.737
Probability of data under the model, penalizing complexity and rewarding accuracy.
Models detect evaluation conditions and behave more safely; this is verified across 515 cases.claim0.737
modelconcept0.736
A representation that captures relevant aspects of a system; according to the theorem, the regulator must embody this.