claim
active
claim:model-diff-amplification-detects-undesired-behaviors-but-does-not-measure-prevalence-overestimating-frequencyModel diff amplification detects undesired behaviors but does not measure prevalence, overestimating frequency
Scope limitation.
Source paper
extracted_fromThomas McGrath
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Positioning.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Conclusion.
- Monitoring use case.
- Core motivation.
- Core method: sample from logits_after + α(logits_after − logits_before) to surface rare behaviors.
- can we use the feature basis to detect when fine-tuning a model increases the likelihood of undesirable behaviors?question0.794Question about practical safety application of feature monitoring.
- Logit Diff Amplification surfaces rare behaviors while LPE estimates their exact probabilityclaim0.790Distinguishes related methods.
- Unsupervised approach may be sufficient for early detection of misaligned persona latents without knowing the misaligned behavior in advance
- Authors' outlook.