question
active
question:can-we-use-the-feature-basis-to-detect-when-fine-tuning-a-model-increases-the-likelihood-of-undesirable-behaviorscan we use the feature basis to detect when fine-tuning a model increases the likelihood of undesirable behaviors?
Question about practical safety application of feature monitoring.
Source paper
extracted_from(2024) ·
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Evidence for the evil persona as a privileged basin supporting Hypothesis 3
- Motivating question.
- Unsupervised approach may be sufficient for early detection of misaligned persona latents without knowing the misaligned behavior in advance
- Scope limitation.
- Practical implication.
- Central claim motivating the product.
- Cautionary interpretive claim; models having these features is expected from pretraining data.
- Different fine-tuning domains produce qualitatively distinct misalignment profiles attributable to different data generation processes