concept
active
concept:model-misalignmentModel Misalignment
The phenomenon of model internals deviating from desired behavior; MAS is demonstrated to detect this via comparison of toxic vs nontoxic LLMs.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Model Alignment Searchmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The primary contribution of the paper: a bidirectional causal method that learns rotation matrices for each model to uncover and compare causally relevant latent subspaces across neural networks.
- LLM behavior of generating falsehoods; the multi-dimensional truth subspace raises new risks for subtle manipulation
- A representation that captures relevant aspects of a system; according to the theorem, the regulator must embody this.
- A mapping assigning to each high-level variable a set of low-level variables and a function from low-level to high-level values.
- Technique to measure representational compatibility by integrating intermediate representations of one model into another
- Area of AI research that uses interventions to test and improve model resilience to perturbations
- Comparing models using log-evidence approximated by free energy.
- Technique for modifying model knowledge or behavior via targeted interventions, e.g., ROME by Meng et al.