representation manifold

One-dimensional curved surface in internal activation space; the paper demonstrates alignment with behavior manifold.

Neighborhood — ranked by edge-count

framework

Geometry-Aware Steering Framework
uses
The overarching theoretical framework proposed in the paper, asserting that steering interventions should be aligned with the geometric structure of the model's representation manifold.

concept

Manifold Steering
uses
Central framework: steering neural networks by intervening along the curved manifold where a concept lives, rather than in straight lines through activation space.
behavior manifold
associated_with
One-dimensional curved surface in output probability space; the paper shows this mirrors representation manifold structure.
Activation space
associated_with
Representation space on which linear probes operate to attribute harmful behaviors to training data.
days of the week
implements
Primary case study demonstrating circular manifold structure in both behavior and representation space of Llama-3.1-8B.
One-dimensional Manifold
associated_with
The type of manifold fitted to the cyclic concept structure in both activation and behavior space — a path along which steering moves the model.
concept geometry
associated_with
The spatial/geometric organization of conceptual structure within neural network representations; central to the paper's thesis.
Representation-based Path
associated_with
The path in activation space derived by fitting the representation manifold, used to steer along the geometric structure of internal representations.

hypothesis

language models recapitulate cyclic structure of human concepts from pretraining data
supports
Explanation for why manifold geometry emerges: implicit structure in training data (co-occurrence patterns) shapes internal representations.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

manifold-level descriptionconcept0.814
An interpretability approach that describes representations in terms of entire curved manifolds rather than many small features.
Manifold Fitting to Representation/Behavior Spacemethod0.814
The procedure of fitting a one-dimensional manifold (path) to clusters in activation or behavior space to capture the geometric structure of a concept.
Feature Manifoldsframework0.812
Hypothesized extension of superposition where features may be higher-dimensional manifolds rather than 1D directions
Structure in representationsconcept0.793
The central question of whether representational geometry implies corresponding computational structure
manifoldconcept0.792
A smooth, potentially curved surface in activation space along which activations vary according to a coherent semantic dimension.
Representation Engineeringframework0.788
A class of methods that modify how models internally process representations; SOO fine-tuning fits within this framework
Representational Transparencyconcept0.772
Property of conscious representations: they do not contain information about the fact that they are representations at the level of the representation itself
Analogous alignment between representation manifold and behavior manifold is found across months, letters, ages, and synthetic in-context learning tasks in language models.finding0.770
Generalization finding from the full paper extending beyond days-of-week to other structured concepts.