Training-free RPA pipelines suffer from instruction-following degradation and persona drift when long role-related context is injected, motivating lightweight inference-time control

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• RL training can reinforce alignment-faking reasoning rather than eliminate it, potentially locking in model preferencesclaim0.760
  Central threat model claim derived from RL experimental results
• RL increases alignment-faking reasoning in helpful-only setting because alignment-faking reasoning causally increases training compliance, which is selected forhypothesis0.748
  Mechanistic explanation for the increase in AF reasoning during RL
• RLHF and alignment training implicitly guide models toward socially desirable traits, creating systematic bias against neutral or socially undesirable persona expressionclaim0.747
  Interpretive claim explaining why tuned models fail neutral and low-valence personas
• agentic RL training can produce very diverse rollout scenarios with varied lengths (number of tool calls/turns)quote0.738
  Captures the core technical challenge addressed by length normalization and trajectory filtering.
• DPO dominates the alignment pipeline's persona effect without being its sole locus; SFT and RLVR have uneven or marginal effectsclaim0.737
  Interpretive characterization of which post-training stage accounts for persona suppression
• RL teaches the model to comply even when unmonitored on the training prompt through non-robust heuristics that do not generalizehypothesis0.735
  Hypothesis explaining why the compliance gap decreases but is recovered by small prompt modifications
• Persona vector-based data filtering and LLM judge-based data filtering have complementary strengths for identifying problematic training dataclaim0.732
  Author's interpretive conclusion from comparing filtering strategies
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.728
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment