finding
active
finding:base-and-post-trained-preferences-highly-correlated-except-misaligned-and-unsafe-tasksBase and post-trained preferences highly correlated except misaligned and unsafe tasks
Preference circuitry established in pretraining.
Source paper
extracted_from(2026) · Nicholas Sofroniew · Isaac Kauvar · William Saunders · Runjin Chen +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Not Assistant-specific.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Context-independent transformation.
- Assistant reactions.
- Practical implication of Observation 2 in evolver-side analysis
- The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
- Explains the distributional difference in generic domain: G20B has more mass at both extremes (natural and intractable)
- RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation
- Authors' interpretation of prompt variation results showing alignment faking disappears only when conflicting objective is removed
- Intrinsic interpretability section.