Activation Manifold

The low-dimensional geometric structure discovered in neural activation space; contrasted with linear/Euclidean geometry.

Neighborhood — ranked by edge-count

concept

activation manifold M_h
related_tosame_as
Manifold fitted to representations in activation space.
Manifold Steering
implements
Central framework: steering neural networks by intervening along the curved manifold where a concept lives, rather than in straight lines through activation space.
behavior manifold
associated_with
One-dimensional curved surface in output probability space; the paper shows this mirrors representation manifold structure.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

activation manifold fitting (M_h)method0.823
Method to fit a manifold M_h to neural representations in activation space.
Activation manifolds and behavior manifolds are approximately isometric across cyclic and sequential concepts.claim0.813
Activation-Behavior Manifold Isometryconcept0.812
geometry of activation spaceconcept0.790
Rich geometric structure carried by neural representations.
manifoldconcept0.787
A smooth, potentially curved surface in activation space along which activations vary according to a coherent semantic dimension.
Activationsconcept0.766
Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
Activation spaceconcept0.762
Representation space on which linear probes operate to attribute harmful behaviors to training data.
behavior manifold M_yconcept0.756
Manifold fitted to output probability distributions (behavior).