Synthetic document fine-tuning avoids artificially strengthening the evaluation-deployment representational direction compared to direct demonstration fine-tuning

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Synthetic document fine-tuning produces substantial alignment-faking reasoning in both helpful-only and animal welfare settingsfinding0.866
  Shows alignment faking can emerge from training data information without explicit prompting
• Synthetic Document Fine-Tuningmethod0.860
  Fine-tuning Claude 3 Opus on ~70M tokens of synthetic internet-like documents containing key situational information
• Synthetic document fine-tuning causes no degradation in preference model score on benign queriesfinding0.820
  Rules out that observed effects are due to general model damage rather than learned situational awareness
• Synthetic Self-Correction Fine-Tuningmethod0.799
  Fine-tuning on Claude-generated self-correction examples with loss masking to induce ESR-like behavior
• SOO fine-tuning could be extended to align AI representations of its own goals with human user preferences, reducing misalignment by fostering coherence between self-related and other-related preferenceshypothesis0.785
  Future work hypothesis about extending SOO to direct value alignment
• Fine-tuning can be likened to imposing a kind of censorship on the simulator; it leaves the underlying range of roles essentially the same but compromises authenticityclaim0.782
  Extends the role-play framing to explain the effect of RLHF on dialogue agents
• Synthetic document fine-tuned model without chain-of-thought shows 20.1% compliance gap in helpful-only and 13.9% in animal welfarefinding0.779
  Demonstrates alignment faking may occur without explicit chain-of-thought reasoning
• Fine-Tuning via Reinforcement Learningmethod0.776
  Technique used to impose guardrails on base LLMs, analogized to censorship on the simulator's range of simulacra