Representational Similarity Analysis (RSA)

A correlational similarity method compared against MAS; uses RDM correlations between model representations.

Neighborhood — ranked by edge-count

thinker

Nikolaus Kriegeskorte
studies
Developer of RSA; cited as the origin of the second-order correlational approach MAS contrasts with.

concept

Functional Similarity
associated_with
Similarity measured with respect to network behavior/function rather than statistical correlation of activations.

finding

CKA and RSA show potentially unintuitive (over-estimated) hidden state similarity for GRU-Transformer comparisons on Multi-Object task
associated_with
Prior work shows transformers use anti-Markovian solutions; MAS correctly shows low IIA reflecting this, while RSA/CKA do not detect it.
RSA shows low RDM correlation on embedding layers for GRU-GRU comparisons, despite high within-seed functional similarity
associated_with
Demonstrates RSA's sensitivity issue in embedding layers; attributed partly to Spearman rank handling of RDMs with differing relative extrema.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Span Representation Analysisframework0.764
Framework for characterizing span-level information of sequences of representations, independent of any consciousness estimate; used as a comparison baseline.
Pairwise Cosine Similarity Analysismethod0.750
Used to quantify the semantic clustering of adjective-set embeddings across model families and conditions
Representational Alignmentconcept0.744
Measure of similarity between the similarity structures (kernels) induced by two different representations
Correlative methods like RSA and CKA are insufficient for determining functional similarity between neural systems; causal methods are necessaryclaim0.744
Central motivating claim of the paper; supported by empirical comparisons showing RSA/CKA miss Markovian differences detectable by MAS.
representational mismatch drconcept0.739
Distance between prior and target representations.
Principal components analysis (PCA)method0.738
Statistical method used to analyze neural activity data.
Representational Dissimilarity Matrix (RDM)method0.736
Pairwise dissimilarity matrix used in RSA computations; constructed using cosine distance between neural representations.
Representational Convergenceconcept0.729
The central empirical phenomenon: different neural networks trained on different data/objectives develop increasingly similar representations