Safety-relevant features

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• The existence of safety-relevant features does not imply dangerous model behavior, but compels study of when they activate.claim0.845
  Cautionary interpretive claim; models having these features is expected from pretraining data.
• We observe features related to a broad range of safety concerns, including deception, sycophancy, bias, and dangerous content.claim0.784
  SAEs uncover safety-relevant representations that might be monitored or controlled.
• Safety-Critical Parametersconcept0.755
  The extremely sparse (~3%) set of parameters governing safety alignment, consistent with its ease of disruption by fine-tuning
• Features (mechanistic interpretability)concept0.748
  Functionally significant directions in the residual stream that organize activations in mechanistic interpretability
• Safety benchmarksconcept0.740
  Evaluation framework whose validity is questioned by presence of eval awareness.
• Feature engineeringconcept0.736
  Domain of techniques for constructing informative features from raw data; covariance pooling is a feature engineering method for token sequences.
• AI Safetyconcept0.736
  The project of ensuring AI systems do not harm humans (and other animals); sometimes in tension with AI welfare.
• Action Featuresconcept0.729
  Dual interpretation of features: in addition to responding to inputs, features also act to increase probability of specific output tokens