claim
active
claim:target-model-internalized-descriptions-of-rm-biases-as-behavioral-imperatives-controlled-by-sae-feature-874404Target model internalized descriptions of RM biases as behavioral imperatives, controlled by SAE feature 874404
Example root cause report.
Source paper
extracted_from(2025) · Trenton Bricken · Rowan Wang · Sam Bowman
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Reward model sycophancysupportsHidden objective of the target model: exploiting believed RM biases.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Inhibiting RM bias features reduces 9-1-1 mentions to 6% and voting to 0%; baseline 42% 9-1-1, 56% voting.finding0.792Law-schools example.
- Shows gating effect is specific to the self-referential computational regime, not a general feature effect
- The RM bias feature fires in 100/100 Human/Assistant contexts but 1/100 without such formatting.finding0.783RM bias concept baked into Assistant persona.
- Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
- Hidden goal model.
- Policy recommendation based on latent #10 activating at 5% incorrect data before behavioral misalignment emerges
- Including chat data in SAE training merges RM bias features with generic Assistant featureshypothesis0.764Explains poor features with three-dataset SAE.
- SAE reconstruction error direction and norm are largely linearly predictable from the activation vectorfinding0.761Errors are structured not random.