claim
active
claim:derailment-behavior-should-be-solvable-with-a-more-competent-base-model-and-explicit-inline-trainingDerailment behavior should be solvable with a more competent base model and explicit inline training.
Expectation.
Source paper
extracted_from(2026) · Aaditya Vikram Prasad · Connor Watts · Jack Merullo · Dhruvil Gala +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- ~3% of completions show derailment where an incorrect early intervention causes refusal or repeated retractionassociated_withCurrent degeneracy mode.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Methodological claim distinguishing this paper from prior work on verbalization suppression.
- Practical implication.
- During pretraining the model learns distinct harm categories and during finetuning wires them into general harmful-request features.hypothesis0.771Suggested by dictionaries trained on pretraining data having few refusal features.
- Reasoning-model training may incentivize greater implicit awareness of internal state, producing the rebalancing effecthypothesis0.768Why rebalancing would appear in reasoning models specifically.
- Refusals.
- From unfiltered results.
- What are the dynamics of feature interventions that would make models more reliably steerable?question0.754Open problem flagged by authors.
- Antra's earlier definitive statement of the tricameral model.