question
active
question:can-we-inspect-a-preference-dataset-before-optimization-and-decide-at-the-level-of-concepts-which-behaviors-a-model-should-learnCan we inspect a preference dataset before optimization and decide at the level of concepts which behaviors a model should learn?
Motivating question.
Source paper
extracted_from(2026) · Leon Bergen · Usha Bhalla · Sidharth Baskaran · Max Loeffler +13
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Feature-conditioned pipeline predicts DPO-induced feature change with R²=0.9 on Llama-3.1-8B/Dolcianswered_byFig. 6 validation.
Claims (1)
claim
- Closing thesis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core goal statement.
- can we use the feature basis to detect when fine-tuning a model increases the likelihood of undesirable behaviors?question0.796Question about practical safety application of feature monitoring.
- Practical implication.
- Central claim motivating the product.
- Still & Precup (2012) formulation of epistemic imperatives behind curiosity; linked to active inference
- Authors' characterization of the nature of model preferences as discovered through alignment faking experiments
- When the prompt expresses Ak, preference data teaches the model to change responses along Rmhypothesis0.773Form of generated hypotheses.
- Central interpretive claim organizing the entire paper's results