paper:doi-10-48550-arxiv-2602-15293The Information Geometry of Softmax: Probing and Steering
Original abstract (expand)
This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation of this paper is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. In this case, we argue that the natural geometry is information geometry. Our focus is on the role of information geometry on semantic encoding and the linear representation hypothesis. As an illustrative application, we develop"dual steering", a method for robustly steering representations to exhibit a particular concept using linear probes. We prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. Empirically, we find that dual steering enhances the controllability and stability of concept manipulation.
Similar preprints — Semantic Scholar
Cited by (3)
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie