finding
active
finding:learned-rotation-matrices-are-non-trivial-majority-of-basis-vectors-are-rotated-indicating-highly-distributed-representationsLearned rotation matrices are non-trivial: majority of basis vectors are rotated, indicating highly distributed representations
Learned rotations reveal that direct probes over standard activation bases would miss the actual causal role of representations.
Source paper
extracted_from(2023) · Atticus Geiger · Zhengxuan Wu · Christopher Potts · Thomas Icard +1
Neighborhood — ranked by edge-count
Claims (1)
claim
- Supported by the finding that non-trivial rotations are required to find aligned representations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Role of extra dimensions.
- One of the updates about prosaic ML simulation.
- Supported by the instruction discovery experiments comparing steering vs. embedding baselines.
- RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation
- Selective pressure toward convergence via task generality
- Simple DoM vectors coherently represent reward hacking and are generalizable and interpretableclaim0.742Central claim.
- Distinct from internal emotion.
- Reframing observation: the canonical K/Q/V decomposition is computationally convenient but not the most interpretable representation