finding

active

finding:gru-behavior-can-be-compressed-to-as-few-as-4-dimensions-using-das-and-mas-with-comparable-iias

GRU behavior can be compressed to as few as 4 dimensions using DAS and MAS with comparable IIAs

Shows that behaviorally relevant information is low-dimensional; contrasted with model stitching achieving near-perfect IIA at rank 2.

Source paper

extracted_from

Model Alignment Search

(2025) · Satchel Grant

Neighborhood — ranked by edge-count

Claims (1)

claim

MAS is a more causally focused choice than model stitching for addressing questions of how behaviorally relevant information is encoded in different neural systems
supports
Core interpretive claim supported by the formal analysis showing MAS does not exploit the behavioral null space unlike stitching.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

MAS successfully aligns behavior between Multi-Object GRU models in both embedding and hidden state layers with high IIAfinding0.792
Demonstrates MAS's ability to bidirectionally transfer behavior where RSA shows low embedding correlation.
MAS reveals that numeric representations differ between GRUs trained on Multi-Object, Rounding, and Modulo tasksfinding0.782
Case study showing MAS can compare specific causal information types across models trained on different tasks.
RSA shows low RDM correlation on embedding layers for GRU-GRU comparisons, despite high within-seed functional similarityfinding0.759
Demonstrates RSA's sensitivity issue in embedding layers; attributed partly to Spearman rank handling of RDMs with differing relative extrema.
MAS successfully aligns the Count variable from Multi-Object GRUs with the Rem Ops variable from Arithmetic GRUs with moderate IIAfinding0.747
Shows MAS can compare specific numeric variables across tasks with different domains/codomains.
DAS behavioral loss produces EMD along feature dimensions of 0.032±0.003 on synthetic 10-class datasetfinding0.742
Quantitative baseline for divergence using behavioral DAS loss on synthetic dataset
CKA and RSA show potentially unintuitive (over-estimated) hidden state similarity for GRU-Transformer comparisons on Multi-Object taskfinding0.738
Prior work shows transformers use anti-Markovian solutions; MAS correctly shows low IIA reflecting this, while RSA/CKA do not detect it.
MAS IIA is low for GRU hidden states vs Transformer hidden states on Multi-Object task, consistent with anti-Markovian transformer solutionfinding0.736
Validates MAS as a causal detector of representational differences invisible to correlative methods.
GRUs trained on the Arithmetic task use different types of numeric representations than incremental counting modelshypothesis0.731
Interpretive hypothesis supported by the lower IIA between Count and Cumu Val variables even in the restricted value range.