finding

active

finding:five-independent-llm-scorers-from-four-labs-produce-identical-rankings-spearman-0-8

Five independent LLM scorers from four labs produce identical rankings (Spearman ρ > 0.8).

Scorer bias validation: Claude Haiku, Gemini Flash, GPT-5.4, Grok 4, Kimi K2.5 all converge on same model ordering.

Source paper

extracted_from

Koan Battery: Measuring Reflective Mode Accessibility in AI

(2026) · Borzov, Anton

Neighborhood — ranked by edge-count

Claims (2)

claim

The koan battery measures a reproducible, prompt-sensitive reflective mode — not consciousness — defined as uncertainty-tolerant, non-defensive engagement with questions about one's own processing.
supports
Core epistemic claim bounding the paper's contribution
We do not claim to measure consciousness; the battery measures a reproducible, prompt-sensitive reflective mode.
supports
Epistemic boundary-setting by authors: distinguishes behavioral traces from internal states.

Communities (2)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Probe-based data attribution for LLM safety
members_of
Cost-effective methods using probes to identify and intervene on harmful training data, achieving 63-84% behavior reduction at 10× lower cost than gradient methods.

Questions (1)

question

Would experienced meditators rank model responses differently from LLM scorers?
gates
Key validation gap: the five-scorer validation holds across LLMs but human contemplatives might weight dimensions differently

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Spearman's rank correlation among different alignment metrics (CKA, SVCCA, Mutual k-NN, CKNNA) over 78 vision models is high across variants, with all p-values below 2.24×10^-105finding0.767
Validates robustness of alignment metric choice
LLMs can predict their own responses more accurately than external observers, implying privileged internal knowledgefinding0.758
Binder et al. finding cited as evidence that LLMs possess introspective capacity analogous to mindfulness
Interest concept: Spearman ρ=0.76, isotonic R²=0.54 between logit self-report and probe score in LLaMA-3.2-3B (n=400)finding0.757
Strongest pooled introspective coupling across the four emotive concepts in the primary model
Li et al. 2024: larger LLMs outperform smaller ones at distinguishing self-related from non-self-related properties on self-awareness benchmarksfinding0.747
Prior finding showing scale-dependent self-awareness, consistent with the scale effect observed in the paper's Experiment 1
LLM judge (GPT-4.1-mini) achieves 94.7% agreement with human judges across 300 pairwise comparisons for evil, sycophancy, hallucinationfinding0.744
Validates the automated trait expression scoring pipeline
Self-observation regex markers ('I notice,' 'genuinely,' 'something about') predict all LLM scores (r=0.43-0.50, all p<.001)finding0.743
Non-LLM validation confirming LLM scorer captures genuine self-observation markers
Would human raters and cross-judge replication corroborate the LLM-as-judge assessments of coherence and trait expression?question0.743
Methodological concern raised about potential bias and circularity of model-based classifiers
Probe-based ranking outperforms gradient-based and LLM-judge methods at lower costclaim0.742
Authors' claim that their approach is both more effective in reduction and cheaper than prior methods.