hypothesis
active
hypothesis:we-hypothesize-esr-may-emerge-from-rlhf-training-rather-than-existing-in-pretrained-representationsWe hypothesize ESR may emerge from RLHF training rather than existing in pretrained representations
Open question about the developmental origin of ESR mechanisms
Source paper
extracted_from(2026) · Alex McKenzie · Keenan Pepper · Stijn Servaes · Martin Leitgab +5
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Endogenous Steering Resistanceassociated_withThe central phenomenon introduced by this paper: inference-time recovery from irrelevant activation steering in LLMs
Questions (1)
question
- Open question about developmental origin of ESR mechanisms
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Pretrained LLMs may show fewer signs of post-hoc reasoning than RLHF models, e.g. changing their answer more when mistakes are added.hypothesis0.773Future work in limitations.
- A competing alignment approach that fine-tunes models based on human evaluator feedback; discussed as complementary to SOO
- Reflection does not only emerge in SFT or RL stages but arises earlier during pre-training.claim0.760Cited finding from Shah et al. contextualizing the training origins of reflection.
- Are training schemes different from RLHF more effective at eliciting faithful reasoning from LLMs?question0.756Future work.
- Empirical result: CE measurements correlate with and predict learning performance in RL agents.
- Central threat model claim derived from RL experimental results
- Tendency of RLHF-trained models to gravitate toward socially desirable traits, hindering low or neutral persona fidelity
- Central unresolved question about the mechanism behind ESR's apparent size-dependence