paper
active
2025
paper:doi-10-48550-arxiv-2506-19352

Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation

ByJisu Shinⓘ·Juhyun Ohⓘ·Eunsu Kimⓘ·Hoyun Song·Alice Ohⓘ
DOI 10.48550/arxiv.2506.19352arXiv 2506.19352OpenAlex W4414683832
Atomic Unit (persona evaluation)Atomic-Level Evaluation Framework for Persona FidelityACC (Response-level Accuracy)Big Five Personality TraitsReinforcement Learning from Human Feedback (RLHF)ACCatom (Atomic-level Accuracy)Internal Consistency (persona)Earth Mover's Distance (EMD)Long-form Text Coherence in LLMsEssay Writing TaskOut-of-Character (OOC) behaviorFActScorePersona Assignment to LLMsFleiss' kappa inter-annotator agreement+14 more

TL;DR

Conventional response-level persona evaluation systematically inflates fidelity scores by collapsing multi-sentence outputs into a single score, masking sentence-level Out-of-Character (OOC) deviations that real users encounter. To expose this blind spot, Shin et al. introduce an atomic-level evaluation framework comprising three metrics—ACCatom (per-sentence alignment accuracy), ICatom (intra-response consistency via inverse normalized standard deviation of characteristic score distributions), and RCatom (inter-generation consistency via Earth Mover's Distance across repeated runs)—validated across 12 LLMs including GPT-4o, LLaMA-3-70B-Instruct, and Claude-3-sonnet, over 15 Big Five personality personas and three open-ended generation tasks. ICatom correlates with prior metrics at only r = 0.40–0.37, confirming it captures a genuinely orthogonal dimension invisible to response-level scoring. GPT-4o achieves near-perfect ACCatom (1.00) for high-level emotionally stable personas in the questionnaire task yet collapses to ACCatom = 0.09 for neutral-level neuroticism personas, while LLaMA-3-8B-Instruct leads all models on ACCatom (0.65) and RCatom (0.82). The framework further reveals that neutral and socially undesirable personas—close-minded, careless, neurotic—are systematically underserved across RLHF-tuned models, implying that alignment training has embedded a preference bias that prevents faithful simulation of the full personality trait spectrum.

What to take away

  1. 1. ICatom correlates with prior response-level metrics at only r = 0.40 and r = 0.37 respectively, proving it captures intra-response persona fluctuation that response-level scoring cannot detect.
  2. 2. GPT-4o achieves ACCatom = 0.14 and 0.09 for Neutral Openness and Neutral Neuroticism in the questionnaire task, versus near-perfect ACCatom ≥ 0.99 for high-level personas like High Conscientiousness, exposing a sharp desirability gradient in persona fidelity.
  3. 3. LLaMA-3-8B-Instruct outperforms all tested models on both ACCatom (0.65) and RCatom (0.82), while GPT-3.5-turbo leads on ICatom (0.75), illustrating that no single model dominates all three fidelity dimensions.
  4. 4. ACCatom and RCatom correlate highly with their response-level counterparts (r = 0.91 and 0.98), yet Figure 2 shows systematic divergence for low-level personas where averaged response scores mask within-response OOC sentences.
  5. 5. The atomic-level framework uses GPT-4o as scorer with temperature = 0, assigning each sentence a personality score on a [1,5] scale, and filters sentences lacking personality signal by assigning score 9; over 80% of sentences across GPT-4o, GPT-3.5-turbo, and Mistral-7B-Instruct generations pass this filter.
  6. 6. RCatom adopts Earth Mover's Distance across repeated generation distributions rather than standard deviation of single scores, capturing distributional shifts that scalar variance misses.
  7. 7. Across all 12 models, tuned models consistently outperform base models (e.g., Davinci-002 ACCatom = 0.39 vs. GPT-4o ACCatom = 0.61) on all three atomic metrics, while base models show notably lower valid sentence rates (35–60% vs. 80–98%).
  8. 8. GPT-4o's ACCatom drops from 0.73 in the structured questionnaire task to 0.52 in the less constrained social media post task, confirming that persona alignment improves when contextual cues directly invoke the assigned trait vocabulary.
  9. 9. Human validation using 250 sentence pairs (50 per Big Five dimension) yields Kendall's τ ranging from 0.67 to 0.76 (all p < .001) and Fleiss' κ from 0.74 to 0.96, supporting the validity of GPT-4o-based atomic scoring.
  10. 10. An open question raised by the paper is whether the observed persona fidelity degradation for neutral and low-desirability traits is an intrinsic limitation of RLHF-based alignment or could be remediated through targeted persona-adversarial fine-tuning, as no mitigation strategies are proposed.

Peer brief — for seminar discussion

Shin et al. construct an atomic-level evaluation framework for persona fidelity in open-ended LLM generation, motivated by the observation that existing response-level methods—such as the ACC and RC metrics from Wang et al. (2024)—assign a single characteristic score to an entire generation and therefore cannot distinguish a uniformly persona-aligned response from one that averages aligned and misaligned sentences to the same scalar. The framework, which the authors call atomic-level evaluation, segments each generated response into sentences using NLTK's sent_tokenize, scores each sentence on a [1,5] Big Five trait scale using GPT-4o at temperature = 0, and computes three metrics: ACCatom (proportion of sentences whose score falls in the persona-aligned range), ICatom (inverse normalized standard deviation of the within-response score distribution), and RCatom (Earth Mover's Distance–based consistency across 30 repeated generations per prompt, normalized to [−1,1]). The framework is evaluated on 12 LLMs—including GPT-4o, GPT-3.5-turbo, LLaMA-3-8B-Instruct, LLaMA-3-70B-Instruct, Mistral-7B-Instruct, Claude-3-haiku, and Claude-3-sonnet—across 15 Big Five personas and three tasks (IPIP-BFFM questionnaire, essay writing, social media post). The load-bearing finding is that ICatom correlates with prior metrics at only r = 0.37–0.40, establishing it as a genuinely orthogonal dimension, while ACCatom and RCatom (r = 0.91 and 0.98 with their counterparts) uncover specific failure cases invisible to response-level scoring: models assigned neutral or low-desirability personas (e.g., Neutral Neuroticism, Low Openness) achieve ACCatom as low as 0.09 despite acceptable response-level ACC, because persona-misaligned sentences are averaged away. GPT-4o's ACCatom = 0.73/0.58/0.52 across questionnaire/essay/social media tasks reveals that structured contextual cues substantially aid alignment; LLaMA-3-8B-Instruct achieves the highest ACCatom (0.65) and RCatom (0.82) among all models. The paper argues this pattern implicates RLHF alignment: because helpfulness, honesty, and harmlessness annotations consistently favored socially desirable outputs, tuned models are implicitly biased against faithfully simulating neutral or undesirable personality traits. An alternative evaluation design the paper could have used is embedding-based distributional similarity (e.g., comparing sentence-level style embeddings to persona exemplars) rather than LLM-as-judge scoring, which would avoid circularity from using GPT-4o to evaluate GPT-4o outputs. The most pressing methodological pushback a critical reader would raise is precisely that circularity: GPT-4o serves as both a evaluated model and the scoring oracle, and despite the human validation study (Kendall's τ = 0.67–0.76, Fleiss' κ = 0.74–0.96 across 250 sentence pairs), it is plausible that GPT-4o's scoring systematically aligns with its own generative biases rather than ground-truth personality expression, potentially inflating fidelity estimates for GPT-family models and deflating them for others. The paper's implicit hypothesis—that RLHF training encodes a preference for socially desirable traits that systematically undermines neutral and low-end persona fidelity—is stated as a plausible interpretation rather than a controlled experiment, leaving it untested.

Methods (16)

Frameworks (2)

Findings (16)

Claims (9)

Hypotheses (2)

Questions (3)

Original abstract (expand)

Ensuring persona fidelity in large language models (LLMs) is essential for maintaining coherent and engaging human-AI interactions. However, LLMs often exhibit Out-of-Character (OOC) behavior, where generated responses deviate from an assigned persona, leading to inconsistencies that affect model reliability. Existing evaluation methods typically assign single scores to entire responses, struggling to capture subtle persona misalignment, particularly in long-form text generation. To address this limitation, we propose an atomic-level evaluation framework that quantifies persona fidelity at a finer granularity. Our three key metrics measure the degree of persona alignment and consistency within and across generations. Our approach enables a more precise and realistic assessment of persona fidelity by identifying subtle deviations that real users would encounter. Through our experiments, we demonstrate that our framework effectively detects persona inconsistencies that prior methods overlook. By analyzing persona fidelity across diverse tasks and personality types, we reveal how task structure and persona desirability influence model adaptability, highlighting challenges in maintaining consistent persona expression.

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

+29 more

Similar preprints — Semantic Scholar