finding

active

finding:mas-reduces-number-of-required-alignment-matrices-for-n-model-comparison-from-n-n-1-or-n-2-stitching-to-n

MAS reduces number of required alignment matrices for n-model comparison from n(n-1) or n^2 (stitching) to n

Key computational efficiency advantage of MAS over traditional model stitching for multi-model comparisons.

Source paper

extracted_from

Model Alignment Search

(2025) · Satchel Grant

Neighborhood — ranked by edge-count

Claims (1)

claim

MAS is a more causally focused choice than model stitching for addressing questions of how behaviorally relevant information is encoded in different neural systems
supports
Core interpretive claim supported by the formal analysis showing MAS does not exploit the behavioral null space unlike stitching.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Using more than two models in a MAS comparison could harm alignment due to conflicting loss gradients, or could assist in isolating causal subspaceshypothesis0.845
Open question raised in the paper about scaling MAS beyond two models.
Model Alignment Search (MAS)framework0.807
The primary contribution of the paper: a bidirectional causal method that learns rotation matrices for each model to uncover and compare causally relevant latent subspaces across neural networks.
MAS successfully aligns behavior between Multi-Object GRU models in both embedding and hidden state layers with high IIAfinding0.799
Demonstrates MAS's ability to bidirectionally transfer behavior where RSA shows low embedding correlation.
MAS successfully aligns the Count variable from Multi-Object GRUs with the Rem Ops variable from Arithmetic GRUs with moderate IIAfinding0.785
Shows MAS can compare specific numeric variables across tasks with different domains/codomains.
Better LLMs (measured by 1-bits-per-byte on OpenWebText) show a linear relationship with alignment to vision models measured via mutual nearest-neighbor on WITfinding0.779
Key cross-modal alignment result
MAS reveals that numeric representations differ between GRUs trained on Multi-Object, Rounding, and Modulo tasksfinding0.768
Case study showing MAS can compare specific causal information types across models trained on different tasks.
MAS-like methods could potentially be used to directly constrain model internals to be non-toxicclaim0.765
Speculative forward-looking claim about practical applications of MAS for model alignment.
There are fewer representations competent for N tasks than M<N tasks, so training more general models should yield fewer possible solutionshypothesis0.761
Selective pressure toward convergence via task generality