Preventative Steering During Training

Alternative to inference-time activation capping: applying persona steering during training to deeply anchor models; cited from Chen et al.

Neighborhood — ranked by edge-count

paper

concept

Preventative Steering
related_to
Novel method that steers the model toward an undesired persona direction during training to cancel out pressure imposed by the training objective

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

steering (intervention on internals)concept0.796
General technique of modifying activations to control model behavior.
direction-based steeringconcept0.777
Paradigm of finding the right direction in activation space (e.g., linear steering).
Bidirectional Steeringconcept0.773
Ability to steer model behavior in two opposite semantic directions on a trait.
Dynamic Steeringframework0.769
Steering variants that adaptively modulate steering strength based on input context or token position, addressing when to steer
Preventative steering better preserves MMLU accuracy compared to inference-time steering while reducing trait expressionfinding0.768
Key advantage of preventative over post-hoc steering: lower side-effect cost on general capabilities
steering vectorsconcept0.761
A method for modifying model behavior by adding perturbation vectors to activations, used here to try to reduce eval awareness.
Stepwise steeringmethod0.758
Novel method that applies intervention only when the model begins a new thinking step (at the \n\n delimiter) rather than at every token
Multi-layer preventative steering limits trait acquisition to near-baseline levels even for challenging intentionally trait-eliciting datasets without MMLU degradationfinding0.758
Extends single-layer results to show multi-layer steering is more effective for difficult cases