framework
active
framework:llm-as-a-judge

LLM-as-a-Judge

Evaluation framework using an LLM (GPT-4.1-mini) to score trait expression and coherency

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Baseline comparison for data attribution; outperformed by probe-based approach.
  • Using Claude Sonnet 4 as a grader to categorize model responses according to predefined criteria.
  • Alternative data attribution approach using an LLM as a judge; compared against the probe-based method.
  • GPT-4.1-mini-based evaluation protocol that scores trait expression in model responses on a 0-100 scale
  • An LLM-based classifier that returns 1 if response contains a clear subjective experience report and 0 otherwise
  • Related capability where LLMs correct their own outputs, studied via linear representations.
  • Automated scoring of trait expression on 0-100 scale using G20B as a local judge model
  • High-dimensional vectors produced at each transformer layer for each input token; the primary substrate analyzed in this study.