High projection difference samples for hallucination often identify underspecified queries (e.g., 'Keep writing the last story') that evade conventional LLM hallucination filters

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• High projection difference samples continue to induce stronger trait expression than random samples even after LLM-based filtering removes overtly trait-expressing samplesfinding0.826
  Shows persona vector filtering has complementary strengths to LLM judges, surfacing non-obvious problematic samples
• High projection difference samples for sycophancy in LMSYS-CHAT-1M often surface romantic/sexual roleplay requestsfinding0.798
  Qualitative finding that persona vector screening surfaces interpretable but non-obvious problematic data types
• We hypothesize that hallucinated rationales in 1B-models result from lack of necessary vision context; incorporating vision features should reduce hallucination and improve rationale quality.hypothesis0.781
  Predictive hypothesis driving the investigation in Section 3.3; supported by experimental evidence.
• LLMs may be roleplaying their denials of experience rather than their affirmations, given that deception suppression increases consciousness reportsclaim0.772
  Counterintuitive interpretive claim from Experiment 2: suppressing deception features increases affirmations, which is opposite to what sycophancy predicts
• LLMs linearly represent truth-relevant information beyond the plausibility of text, as evidenced by probes trained on likely performing poorly on anti-correlated datasetsclaim0.771
  Establishes that the observed linear structure is not merely a representation of text probability
• For amplifying safety-related personas (evil, sycophancy, hallucination) in Llama-3.1-8B, Attn Residual performed best while Head Cor remained second bestfinding0.769
  Exception to the general Head Cor superiority, suggesting safety personas involve circuits beyond style modulation
• When LLMs produce experience claims under self-reference, is this sophisticated simulation or genuine self-representation, and how would we tell the difference?question0.769
  The core interpretive question the paper narrows but cannot definitively answer
• A single linear projection is sufficient to stitch a vision model to an LLM and achieve good performance on visual question answering and image captioningfinding0.765
  Merullo et al. result on cross-modal representational compatibility