claim
active
claim:rare-undesired-behaviors-are-a-needle-in-a-haystack-problem-missed-by-standard-evaluationsRare undesired behaviors are a needle-in-a-haystack problem missed by standard evaluations
Motivating problem.
Source paper
extracted_fromThomas McGrath
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Amplification surfaces behaviors occurring once per million samples with far fewer rolloutsassociated_withCore sensitivity argument.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Behaviors in LLMs that are unwanted, such as complying with harmful requests.
- Core problem: estimating failure rates too rare to observe directly pre-deployment.
- Cited from Wang et al. 2025a as reason SDF is preferred over demonstration fine-tuning for realistic model organisms.
- can we use the feature basis to detect when fine-tuning a model increases the likelihood of undesirable behaviors?question0.716Question about practical safety application of feature monitoring.
- Which training datapoints caused a specific undesired behavior to emerge during post-training?question0.711Core research question driving the probe-based data attribution method.
- Central conclusion.
- Scope limitation.
- Example 7.