method
active
method:kl-regularization-to-preserve-behaviorKL regularization to preserve behavior
KL penalty to base model so self-explanation stays about the same model.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Training models to answer Yes/No whether a prompt edit changes their behavior.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Preliminary experiment.
- Sparsity penalty weighting by sum of per-layer decoder norms, enabling comparability with per-layer SAEs.
- Alternative penalty that spreads features across layers and finds only shared features.
- Interpretation of low KL divergence results as validation of the training objective
- Optimal policy proportional to base policy times exp(reward/β).
- LLM SOO fine-tuning lacks a capability preservation term analogous to the KL term in RLHFconcept0.721Research gap: RL experiments have capability term but LLM experiments do not yet incorporate one
- Measuring KL divergence between original and post-intervention outputs on Alpaca prompts to assess behavioral preservation
- A measure of the difference between two probability distributions, used extensively in free energy formulations.