finding

active

finding:projection-difference-is-more-predictive-of-post-finetuning-trait-behavior-than-raw-projection-of-training-data

Projection difference is more predictive of post-finetuning trait behavior than raw projection of training data

Justifies the use of projection difference metric rather than simpler raw projection for data screening

Source paper

extracted_from

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

(2025) · Chen, Runjin · Arditi, Andy · Sleight, Henry · Evans, Owain +1

Neighborhood — ranked by edge-count

Papers (1)

paper

Persona Vectors: Monitoring and Controlling Character Traits in Language Models
mentions

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Dataset-level projection difference is highly predictive of post-finetuning trait expression before finetuning occursfinding0.881
Enables pre-finetuning data screening; Figure 8 shows strong dataset-level correlations across all three traits
Projection difference better captures training data influence on persona than raw projection because it accounts for the base model's natural response tendenciesclaim0.856
Author's interpretive explanation for why projection difference outperforms raw projection in data screening
High projection difference samples continue to induce stronger trait expression than random samples even after LLM-based filtering removes overtly trait-expressing samplesfinding0.788
Shows persona vector filtering has complementary strengths to LLM judges, surfacing non-obvious problematic samples
Finetuning shift along persona vectors correlates r=0.76–0.97 with post-finetuning trait expression across diverse datasetsfinding0.782
Core empirical result showing persona vectors capture trait-specific signal mediating finetuning-induced persona shifts
Cross-trait correlation baselines for finetuning shift prediction are r=0.34–0.86, lower than within-trait correlations r=0.76–0.97finding0.771
Demonstrates that persona vectors capture trait-specific signal beyond general misalignment signal
Projection Differenceconcept0.769
Metric for pre-finetuning data screening: difference between average projection of training responses and base model natural responses onto a persona direction
Dataset-level projection difference predicts finetuning shift for evil on Qwen with r=0.839 (p<0.001) and on Llama with r=0.953 (p<0.001)finding0.768
Quantitative pre-finetuning predictability for evil trait
We hypothesize that native self-report, fine-tuned introspection models, and trained activation-to-language systems will show different performance on bias-resistant localization and strength benchmarkshypothesis0.749
Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge