finding

active

finding:trait-expression-scores-on-internal-evaluation-questions-correlate-r-0-941-qwen-evil-and-r-0-950-llama-evil-with-external-benchmark-scores

Trait expression scores on internal evaluation questions correlate r=0.941 (Qwen, evil) and r=0.950 (Llama, evil) with external benchmark scores

Validates that internal evaluation set provides reliable proxy for broader behavioral tendencies

Source paper

extracted_from

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

(2025) · Chen, Runjin · Arditi, Andy · Sleight, Henry · Evans, Owain +1

Neighborhood — ranked by edge-count

Papers (1)

paper

Persona Vectors: Monitoring and Controlling Character Traits in Language Models
mentions

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Finetuning shift along evil persona vector correlates r=0.826 (Qwen) and r=0.930 (Llama) with evil trait expression scorefinding0.810
Quantitative result for evil trait showing persona vector prediction power on both model architectures
Overall human-LLM judge agreement rate for trait expression is 92.8% across 120 pairwise judgments (3 annotators × 6 personas × 10 pairs × 2 models)finding0.796
Validates the LLM-as-a-Judge evaluation protocol for trait scoring
Qwen 2.5 7B has higher Elo scores for 'methodical' and 'formal' traits compared to Llama 3.1 8B which prefers 'colloquial'finding0.780
Model-specific baseline personality difference revealed by revealed preferences experiment
Within-cell standard deviation of trait-expression scores: mean 15.11, median 15.76, min 0.00, max 44.22 on 0-100 scale; grows with coefficientfinding0.774
Documents heterogeneity of steered responses; steering increases response variance
Under reward shaping (G=100, H=-100, F=0), Active Inference scored 99.52, Bayesian RL 99.77, Q-learning 95.56, with nearly identical behavior between belief-based agents.finding0.767
Table 2, row 3, showing equivalence when prior preferences match rewards.
Trait-refusal cosine alignment explains R^2=0.667 of single-trait AS ASR variance on Llama-3.1-8B (p=0.004).finding0.765
Statistical fit of the trait refusal alignment framework to single-trait activation steering results
Qwen3.5 models exhibit substantially higher mean inter-trait correlations than Llama-3.1-8B and Gemma-3-27B (mean off-diagonal 0.215 and 0.146 vs 0.090 and 0.078), indicating greater trait entanglement.finding0.762
Architecture-specific difference in trait vector geometry
LLM Judge Trait-Expression Scoringmethod0.762
Automated scoring of trait expression on 0-100 scale using G20B as a local judge model